Stand-Alone Two-Node HA K3s Cluster
Project Description
Edge environments need highly available infrastructure but due to scale, they also need to reduce costs wherever possible. Lowering compute hardware costs by 1/3 can be the difference between success and failure in launching an edge Kubernetes offering.
This project looks to create a reliable, two node, stand-alone, highly-available K3s cluster.
While doing this completely stand-alone may not be possible, part of the goal is to minimize the external dependencies as much as possible. Those would be things like shared storage, external network resources (including external "witness" services), network-accessed databases and/or backup/synchronization targets, etc.
Currently there are three designs being considered: the etcd design exists in a rough draft. There has been basic framework testing for Keepalived, but little has been documented. NATS/Jetstream remains in an early stage of ideation.
Goal for this Hackweek
The goal will be to create at least one design to meet the description. There are currently three possible designs under consideration: 1) etcd running outside of K3s 2) A SQL database with HA functionality orchestrated through Keepalived 3) NATS/Jetstream
If we have the capacity, even better would be to build out more than one design to allow for head-to-head comparison.
Resources
Very helpful would be people who: - Can write clean shell scripts (There is a lot of interconnected logic in managing different node and failure states) - Will to perform organized iteration testing and document the results - Can write good documentation on how to replicate the designs - Understand SQL and can help integrate it into shell scripts
K3s edge Kubernetes HA
Looking for hackers with the skills:
Nothing? Add some keywords!
This project is part of:
Hack Week 22
Activity
Comments
-
almost 2 years ago by mak3ra | Reply
2 Node HA K8s exists thanks to kine which is embedded in K3s. What would be really interesting here is a 2 node HA distribution of K3s. Currently most K8s distributions have a way to configure and setup ETCD for the user. This is why we use distributions. What would be valuable here is to have a distro that allows the user to setup an HA 2 node cluster as easily as k3s does today. Also important would be to maintain many of the k3s features like "lightweight", or composition of the control plane and data plane into a single executable, or simplified communication through tunneling between worker and control plane, etc..
-
almost 2 years ago by alex.arnoldy | Reply
Thank you for your comment. I can definitely see this becoming the standard for K3s. You simply select one, two, or three+ nodes and K3s does the rest. I've edited the project to emphasis the stand-alone goal of this project. After all, currently a two node, HA K3s cluster actually consists of two nodes, plus some fraction of a complex, HA database as well as the HA networking between them. If the clients are very close, or can tolerate network interruptions, you still need a high quality network all the way to your database. The goal of this project is to reduce or eliminate those external dependencies/costs.
-
-
almost 2 years ago by alex.arnoldy | Reply
As a side note, my interest in this project comes from work I've done recently with two organizations. One of which chose embedded etcd over an external database for their edge design and one who refused both; insisting that a two node, stand-alone, HA K3s is the only thing that meets their needs.
-
almost 2 years ago by soibam | Reply
I would like to contribute to this project with my limited k3s knowledge. my interest in this project - learn and contribute. Comfortable with scripting - shell/python etc.
-
almost 2 years ago by alex.arnoldy | Reply
Great! I'll include you on the kick-off meeting Monday. Please email me at alex.arnoldy@suse.com with your time zone.
-
-
almost 2 years ago by alex.arnoldy | Reply
We'll have a kick off meeting Monday morning to get things rolling. Please email me at alex.arnoldy@suse.com if you haven't received an invite and want to be included.
-
Similar Projects
This project is one of its kind!