Project Description

Edge environments need highly available infrastructure but due to scale, they also need to reduce costs wherever possible. Lowering compute hardware costs by 1/3 can be the difference between success and failure in launching an edge Kubernetes offering.

This project looks to create a reliable, two node, stand-alone, highly-available K3s cluster.

While doing this completely stand-alone may not be possible, part of the goal is to minimize the external dependencies as much as possible. Those would be things like shared storage, external network resources (including external "witness" services), network-accessed databases and/or backup/synchronization targets, etc.

Currently there are three designs being considered: the etcd design exists in a rough draft. There has been basic framework testing for Keepalived, but little has been documented. NATS/Jetstream remains in an early stage of ideation.

Goal for this Hackweek

The goal will be to create at least one design to meet the description. There are currently three possible designs under consideration: 1) etcd running outside of K3s 2) A SQL database with HA functionality orchestrated through Keepalived 3) NATS/Jetstream

If we have the capacity, even better would be to build out more than one design to allow for head-to-head comparison.

Resources

Very helpful would be people who: - Can write clean shell scripts (There is a lot of interconnected logic in managing different node and failure states) - Will to perform organized iteration testing and document the results - Can write good documentation on how to replicate the designs - Understand SQL and can help integrate it into shell scripts

K3s edge Kubernetes HA

Looking for hackers with the skills:

Nothing? Add some keywords!

This project is part of:

Hack Week 22

Activity

  • 4 months ago: rishiy joined this project.
  • 4 months ago: rajeshv joined this project.
  • 4 months ago: okurz liked this project.
  • 4 months ago: fgiudici liked this project.
  • 4 months ago: soibam joined this project.
  • 5 months ago: atgracey joined this project.
  • 5 months ago: roxenham started this project.
  • 5 months ago: alex.arnoldy originated this project.

  • Comments

    • mak3ra
      5 months ago by mak3ra | Reply

      2 Node HA K8s exists thanks to kine which is embedded in K3s. What would be really interesting here is a 2 node HA distribution of K3s. Currently most K8s distributions have a way to configure and setup ETCD for the user. This is why we use distributions. What would be valuable here is to have a distro that allows the user to setup an HA 2 node cluster as easily as k3s does today. Also important would be to maintain many of the k3s features like "lightweight", or composition of the control plane and data plane into a single executable, or simplified communication through tunneling between worker and control plane, etc..

      • alex.arnoldy
        5 months ago by alex.arnoldy | Reply

        Thank you for your comment. I can definitely see this becoming the standard for K3s. You simply select one, two, or three+ nodes and K3s does the rest. I've edited the project to emphasis the stand-alone goal of this project. After all, currently a two node, HA K3s cluster actually consists of two nodes, plus some fraction of a complex, HA database as well as the HA networking between them. If the clients are very close, or can tolerate network interruptions, you still need a high quality network all the way to your database. The goal of this project is to reduce or eliminate those external dependencies/costs.

    • alex.arnoldy
      5 months ago by alex.arnoldy | Reply

      As a side note, my interest in this project comes from work I've done recently with two organizations. One of which chose embedded etcd over an external database for their edge design and one who refused both; insisting that a two node, stand-alone, HA K3s is the only thing that meets their needs.

    • soibam
      4 months ago by soibam | Reply

      I would like to contribute to this project with my limited k3s knowledge. my interest in this project - learn and contribute. Comfortable with scripting - shell/python etc.

      • alex.arnoldy
        4 months ago by alex.arnoldy | Reply

        Great! I'll include you on the kick-off meeting Monday. Please email me at alex.arnoldy@suse.com with your time zone.

    • alex.arnoldy
      4 months ago by alex.arnoldy | Reply

      We'll have a kick off meeting Monday morning to get things rolling. Please email me at alex.arnoldy@suse.com if you haven't received an invite and want to be included.

    • rishiy
      4 months ago by rishiy | Reply

      Hi , I would like to understand the architecture & design for the 3 proposals. Can you please send me the invite too.

    Similar Projects

    This project is one of its kind!