SUSE Hack Week: Small healthcheck tool for Longhorn

Project Description

We have often problems (e.g. pods not starting) that are related to PVCs not running, cluster (nodes) not all up or deployments not running or completely running. This all prevents administration activities. Having something that can regular be run to validate the status of the cluster would be helpful, and not as of today do a lot of manual tasks.

As addition (read enough time), we could add changing reservation, adding new disks, etc. --> This didn't made it. But the scripts can easily be adopted.

This tool would decrease troubleshooting time, giving admins rights to the rancher GUI and could be used in automation.

Goal for this Hackweek

At the end we should have a small python tool that is doing a (very) basic health check on nodes, deployments and PVCs. First attempt was to make it in golang, but that was taking to much time.

Overview

This tool will run a simple healthcheck on a kubernetes cluster. It will perform the following actions:

node check: This will check all nodes, and display the status and the k3s version. If the status of the nodes is not "Ready" (this should be only reported), the cluster will be reported as having problems
deployment check: This check will list all deployments, and display the number of expected replicas and the used replica. If there are unused replicas this will be displayed. The cluster will be reported as having problems.
pvc check: This check will list of all pvc's, and display the status and the robustness. If the robustness is not "Healthy", the cluster will be reported as having problems.

If there is a problem registered in the checks, there will be a warning that the cluster is not healthy and the program will exit with 1.

The script has 1 mandatory parameter and that is the kubeconf of the cluster or of a node off the cluster.

The code is writen for Python 3.11, but will also work on 3.6 (the default with SLES15.x). There is a venv present that will contain all needed packages. Also, the script can be run on the cluster itself or any other linux server.

Installation

To install this project, perform the following steps:

Create the directory /opt/k8s-check

mkdir /opt/k8s-check

Copy all the file to this directory and make the following changes:

chmod +x k8s-check.py

note:

If you want to run this program in you on python environment or the default of the server, please perform the following actions:

install the needed modules:

pip install tabulate kubernetes

change the first line of k8s-check.py to:

\#!/usr/bin/env python3

Usage

As mentioned above, for the executing of the script the kubeconfig is needed. If this happens on the node, please use the local kubeconfig (e.g. for k3s this will be: /etc/rancher/k3s/k3s.yaml). If the script is run on a different server, copy the kubeconfig to this server. Executing the following will do the magic:

/opt/k8s-check --kubeconfig

repository link

https://github.com/mbrookhuis/k8s-check

keywords: longhorn monitoring python

Join this project Leave this project

Looking for hackers with the skills:

kubernetes python3 rancher slemicro edge

This project is part of:

Hack Week 24

Activity

1 day ago: mbrookhuis added keyword "python3" to this project.

1 day ago: mbrookhuis added keyword "rancher" to this project.

1 day ago: mbrookhuis added keyword "slemicro" to this project.

1 day ago: mbrookhuis added keyword "edge" to this project.

1 day ago: mbrookhuis added keyword "kubernetes" to this project.

about 2 months ago: bkampen liked this project.

3 months ago: mbrookhuis started this project.

3 months ago: mbrookhuis originated this project.

Comments

Be the first to comment!

Similar Projects

kubernetes

ddflare: (Dynamic)DNS management via Cloudflare API in Kubernetes by fgiudici

Description

ddflare is a project started a couple of weeks ago to provide DDNS management using v4 Cloudflare APIs: Cloudflare offers management via APIs and access tokens, so it is possible to register a domain and implement a DynDNS client without any other external service but their API.

Since ddflare allows to set any IP to any domain name, one could manage multiple A and ALIAS domain records. Wouldn't be cool to allow full DNS control from the project and integrate it with your Kubernetes cluster?

Goals

Main goals are:

add containerized image for ddflare
extend ddflare to be able to add and remove DNS records (and not just update existing ones)
add documentation, covering also a sample pod deployment for Kubernetes
write a ddflare Kubernetes operator to enable domain management via Kubernetes resources (using kubebuilder)

Available tasks and improvements tracked on ddflare github.

Resources

https://github.com/fgiudici/ddflare
https://developers.cloudflare.com/api/
https://book.kubebuilder.io

A CLI for Harvester by mohamed.belgaied

[comment]: # Harvester does not officially come with a CLI tool, the user is supposed to interact with Harvester mostly through the UI [comment]: # Though it is theoretically possible to use kubectl to interact with Harvester, the manipulation of Kubevirt YAML objects is absolutely not user friendly. [comment]: # Inspired by tools like multipass from Canonical to easily and rapidly create one of multiple VMs, I began the development of Harvester CLI. Currently, it works but Harvester CLI needs some love to be up-to-date with Harvester v1.0.2 and needs some bug fixes and improvements as well.

Project Description

Harvester CLI is a command line interface tool written in Go, designed to simplify interfacing with a Harvester cluster as a user. It is especially useful for testing purposes as you can easily and rapidly create VMs in Harvester by providing a simple command such as: harvester vm create my-vm --count 5 to create 5 VMs named my-vm-01 to my-vm-05.

Harvester CLI is functional but needs a number of improvements: up-to-date functionality with Harvester v1.0.2 (some minor issues right now), modifying the default behaviour to create an opensuse VM instead of an ubuntu VM, solve some bugs, etc.

Github Repo for Harvester CLI: https://github.com/belgaied2/harvester-cli

Done in previous Hackweeks

Create a Github actions pipeline to automatically integrate Harvester CLI to Homebrew repositories: DONE
Automatically package Harvester CLI for OpenSUSE / Redhat RPMs or DEBs: DONE

Goal for this Hackweek

The goal for this Hackweek is to bring Harvester CLI up-to-speed with latest Harvester versions (v1.3.X and v1.4.X), and improve the code quality as well as implement some simple features and bug fixes.

Some nice additions might be: * Improve handling of namespaced objects * Add features, such as network management or Load Balancer creation ? * Add more unit tests and, why not, e2e tests * Improve CI * Improve the overall code quality * Test the program and create issues for it

Issue list is here: https://github.com/belgaied2/harvester-cli/issues

Resources

The project is written in Go, and using client-go the Kubernetes Go Client libraries to communicate with the Harvester API (which is Kubernetes in fact). Welcome contributions are:

Testing it and creating issues
Documentation
Go code improvement

What you might learn

Harvester CLI might be interesting to you if you want to learn more about:

GitHub Actions
Harvester as a SUSE Product
Go programming language
Kubernetes API

Multi-pod, autoscalable Elixir application in Kubernetes using K8s resources by socon

Description

Elixir / Erlang use their own solutions to create clusters that work together. Kubernetes provide its own orchestration. Due to the nature of the BEAM, it looks a very promising technology for applications that run in Kubernetes and requite to be always on, specifically if they are created as web pages using Phoenix.

Goals

Investigate and provide solutions that work in Phoenix LiveView using Kubernetes resources, so a multi-pod application can be used
Provide an end to end example that creates and deploy a container from source code.

Resources

https://github.com/dwyl/phoenix-liveview-counter-tutorial https://github.com/propedeutica/elixir-k8s-counter

Extending KubeVirtBMC's capability by adding Redfish support by zchang

Description

In Hack Week 23, we delivered a project called KubeBMC (renamed to KubeVirtBMC now), which brings the good old-fashioned IPMI ways to manage virtual machines running on KubeVirt-powered clusters. This opens the possibility of integrating existing bare-metal provisioning solutions like Tinkerbell with virtualized environments. We even received an inquiry about transferring the project to the KubeVirt organization. So, a proposal was filed, which was accepted by the KubeVirt community, and the project was renamed after that. We have many tasks on our to-do list. Some of them are administrative tasks; some are feature-related. One of the most requested features is Redfish support.

Goals

Extend the capability of KubeVirtBMC by adding Redfish support. Currently, the virtbmc component only exposes IPMI endpoints. We need to implement another simulator to expose Redfish endpoints, as we did with the IPMI module. We aim at a basic set of functionalities:

Power management
Boot device selection
Virtual media mount (this one is not so basic )

Resources

DMTF/Redfish-Interface-Emulator: The Redfish Interface Emulator can emulate a Redfish-based interface statically (GET) or dynamically (POST, PATCH, DELETE)

Harvester Packer Plugin by mrohrich

Description

Hashicorp Packer is an automation tool that allows automatic customized VM image builds - assuming the user has a virtualization tool at their disposal. To make use of Harvester as such a virtualization tool a plugin for Packer needs to be written. With this plugin users could make use of their Harvester cluster to build customized VM images, something they likely want to do if they have a Harvester cluster.

Goals

Write a Packer plugin bridging the gap between Harvester and Packer. Users should be able to create customized VM images using Packer and Harvester with no need to utilize another virtualization platform.

Resources

Hashicorp documentation for building custom plugins for Packer https://developer.hashicorp.com/packer/docs/plugins/creation/custom-builders

Source repository of the Harvester Packer plugin https://github.com/m-ildefons/harvester-packer-plugin

python3

Selenium with Python by xguo

Description

Try to create test case about Selenium base on Python

Goals

Knowledge about Selenium with Python
Create new test case about Selenium

Resources

https://selenium-python.readthedocs.io/ https://www.selenium.dev/

Symbol Relations by hli

Description

There are tools to build function call graphs based on parsing source code, for example, cscope.

This project aims to achieve a similar goal by directly parsing the disasembly (i.e. objdump) of a compiled binary. The assembly code is what the CPU sees, therefore more "direct". This may be useful in certain scenarios, such as gdb/crash debugging.

Detailed description and Demos can be found in the README file:

Supports x86 for now (because my customers only use x86 machines), but support for other architectures can be added easily.

Tested with python3.6

Goals

Any comments are welcome.

Resources

https://github.com/lhb-cafe/SymbolRelations

symrellib.py: mplements the symbol relation graph and the disassembly parser

symrel_tracer*.py: implements tracing (-t option)

symrel.py: "cli parser"

rancher

Enabling Rancher as an OIDC Provider by rcabello

Description

Kubernetes supports OpenID Connect (OIDC) natively as an authentication mechanism, enabling token-based user authentication. This can be configured through flags in the Kubernetes API server or by using AuthenticationConfiguration.

The purpose of this project is to enable Rancher to function as an OIDC provider, allowing Rancher's local cluster to act as an OIDC identity provider for downstream clusters. This setup will allow users to authenticate directly with downstream clusters without relying on Rancher’s proxy and impersonation mechanisms.

Rancher will continue to support all authentication providers. When a user attempts to log in via the Rancher OIDC provider, they will be redirected to the authentication provider configured in Rancher.

This approach also facilitates integration with third-party tools (e.g StackState)

Goals

Implement Rancher as an OIDC provider using the ORY Fosite library, focusing only on the essential functionality required for basic integration.
Enable downstream clusters to authenticate using JWT tokens issued by Rancher.
Configure StackState to authenticate using Rancher as an OIDC provider.

Resources

https://github.com/ory/fosite

Integrate Backstage with Rancher Manager by nwmacd

Description

Backstage (backstage.io) is an open-source, CNCF project that allows you to create your own developer portal. There are many plugins for Backstage.

This could be a great compliment to Rancher Manager.

Goals

Learn and experiment with Backstage and look at how this could be integrated with Rancher Manager. Goal is to have some kind of integration completed in this Hack week.

Progress

Screen shot of home page at the end of Hackweek:

Day One

Got Backstage running locally, understanding configuration with HTTPs.
Got Backstage embedded in an IFRAME inside of Rancher
Added content into the software catalog (see: https://backstage.io/docs/features/techdocs/getting-started/)
Understood more about the entity model

Day Two

Connected Backstage to the Rancher local cluster and configured the Kubernetes plugin.
Created Rancher theme to make the light theme more consistent with Rancher

Days Three and Day Four

Created two backend plugins for Backstage:
1. Catalog Entity Provider - this imports users from Rancher into Backstage
2. Auth Provider - uses the proxied sign-in pattern to check the Rancher session cookie, to user that to authenticate the user with Rancher and then log them into Backstage by connecting this to the imported User entity from the catalog entity provider plugin.
With this in place, you can single-sign-on between Rancher and Backstage when it is deployed within Rancher. Note this is only when running locally for development at present

Day Five

Start to build out a production deployment for all of the above
Made some progress, but hit issues with the authentication and proxying when running proxied within Rancher, which needs further investigation

A CLI for Harvester by mohamed.belgaied

Project Description

Github Repo for Harvester CLI: https://github.com/belgaied2/harvester-cli

Done in previous Hackweeks

Create a Github actions pipeline to automatically integrate Harvester CLI to Homebrew repositories: DONE
Automatically package Harvester CLI for OpenSUSE / Redhat RPMs or DEBs: DONE

Goal for this Hackweek

Issue list is here: https://github.com/belgaied2/harvester-cli/issues

Resources

The project is written in Go, and using client-go the Kubernetes Go Client libraries to communicate with the Harvester API (which is Kubernetes in fact). Welcome contributions are:

Testing it and creating issues
Documentation
Go code improvement

What you might learn

Harvester CLI might be interesting to you if you want to learn more about:

GitHub Actions
Harvester as a SUSE Product
Go programming language
Kubernetes API

Longhorn UI Extension (POC) by yiya.chen

Description

The goal is to create a Longhorn UI extension within Rancher using existing resources.
Longhorn’s UI is built using React, while Rancher’s UI extensions are built using Vue. Developers will explore different approaches to integrate and extend Longhorn’s UI within Rancher’s Vue-based ecosystem, aiming to create a seamless, functional UI extension.

Goals

Build a Longhorn UI extension (look and feel)
Support theme switching to align with Rancher’s UI

Results

https://github.com/a110605/longhorn-hackday
https://github.com/a110605/longhorn-ui/tree/darkmode
https://github.com/houhoucoop/hackweek/tree/main/hackweek24

Resources

Longhorn UI: https://github.com/longhorn/longhorn-ui
Rancher UI Extension: https://extensions.rancher.io/extensions/next/home
darkreader: https://www.npmjs.com/package/darkreader
veaury: https://github.com/gloriasoft/veaury
module federation: https://webpack.js.org/concepts/module-federation/

Rancher microfrontend extensions by ftorchia

Description

Rancher UI Extensions allow users, developers, partners, and customers to extend and enhance the Rancher UI. Extensions are Helm charts that can only be installed once into a cluster. The charts contain a UI built package that is downloaded and linked to the Host UI at runtime; this means that the extension pkg needs to be implemented using the same technology and have the same APIs as Rancher UI.

Goals

We want to create a new type of Rancher extension, based on microfrontend pattern. The extension is served in a docker container in the k8s clusters and embedded in the host UI; this would guarantee us to be able to create extensions unrelated to the rancher UI architecture, in any technology.

Non Goals

We want to apply the microfrontend pattern to the product-level extensions; we don't want to apply it to cluster-level extensions.

Resources

rancher-extension-microfrontend, Rancher extensions

edge

Build Edge Image Builder ISO with SUSE Manager by mweiss2

Description

With SUSE Manager, we can build OS Images using KIWI and container images. As we have Edge Image Builder, we want to see if it is possible to use SUSE Manager to build/customize OS Images by integrating Edge Image Builder as well.

Goals

To make the process easier for customers, a single-build pipeline that automatically adds the combustion and artifact files from the EIB process is desirable.

Kiwi and EIB need to come from a Git Repository.
Kiwi and EIB need to be running as containers.
Configuration options for the images used for Kiwi and EIB build.
X86 and ARM64 Support.
SUSE Manager 4.3 and 5.X Support.
SLES 15 SP6 / SL Micro 6.0 and SL Micro 6.1 Support.

Outcome

Change the Kiwi build process to use Podman with the Kiwi image registry.suse.com/bci/kiwi:10.1.10
Change the Edge Image Builder to produce a combustion-only ISO
Extract the contents and write them to a dedicated /OEM partition integrated via Kiwi into the ISO Kiwi creates.

Sources and PRs

https://github.com/Martin-Weiss/kiwi-image-micro-gpu-60
https://github.com/suse-edge/edge-image-builder/pull/618
https://github.com/uyuni-project/uyuni/pull/9507

Edge Image Builder and mkosi for Uyuni by oholecek

Description

One part of Uyuni system management tool is ability to build custom images. Currently Uyuni supports only Kiwi image builder.

Kiwi however is not the only image building system out there and with the goal to also become familiar with other systems, this projects aim to add support for Edge Image builder and systemd's mkosi systems.

Goals

Uyuni is able to

provision EIB and mkosi build hosts
build EIB and mkosi images and store them

Resources

Uyuni - https://github.com/uyuni-project/uyuni
Edge Image builder - https://github.com/suse-edge/edge-image-builder
mkosi - https://github.com/systemd/mkosi