Project Description

We know that Kubernetes clusters need control planes but running them in dedicated VMs might not be always efficient, instead, we can run them as pods within the management cluster. This project aims to solve the following problems:

• Reduce the cost of provisioning control planes
• Provide a declarative API for control plane management
• Centralize management of control planes and decouple them from workers
• Provide k3s clusters as a service on any infrastructure.

Goal for this Hackweek

• Look into existing solutions like https://github.com/clastix/kamaji
• Figure out if any can be reused for managing K3S
• Investigate what changes need to be done in K3S in order to support this use-case
• Build an API that allows provisioning and configuring of K3S control planes, see https://github.com/zawachte/cluster-api-k3s/ for generating k3s configuration using K8S API.

Resources

No project repository for now, all hacking will be done in these forks: https://github.com/alexander-demicev/kamaji https://github.com/alexander-demicev/k3s

What was achieved during hack week?

• I was able to deploy k3s in a pod using experimental agentless feature https://docs.k3s.io/advanced#running-agentless-servers-experimental, meaning the server will not run kubelet, container runtime, or CNI
• It was possible to deploy an external etcd and connect agentless servers to it https://docs.k3s.io/installation/ha#2-launch-server-nodes
• I managed to run and connect a worker node to control planes running in pods

What wasn't done during hack week?

• After some investigation I came to conclusion that kamaji might be reused but with some changes to its codebase as we are plugging our kubernetes distro
• I was working on POC operator based on CAPI k3s provider https://github.com/zawachte/cluster-api-k3s/, the operator would manage pod deployments instead of CAPI machines but one week is not enough to get it working