SUSE Hack Week: monitor kubernetes and docker with prometheus

Starting from prometheus ( and grafana if needed), learn how to monitor kubernetes and docker and do some valid alert/graph etc.

https://docs.docker.com/config/thirdparty/prometheus/

Looking for hackers with the skills:

golang prometheus monitoring kubernetes docker grafana

This project is part of:

Hack Week 17

Activity

over 6 years ago: dmaiocchi added keyword "grafana" to this project.

over 6 years ago: dmaiocchi added keyword "golang" to this project.

over 6 years ago: dmaiocchi added keyword "prometheus" to this project.

over 6 years ago: dmaiocchi added keyword "monitoring" to this project.

over 6 years ago: dmaiocchi added keyword "kubernetes" to this project.

over 6 years ago: dmaiocchi added keyword "docker" to this project.

over 6 years ago: dmaiocchi started this project.

over 6 years ago: dmaiocchi originated this project.

Comments

Be the first to comment!

Similar Projects

golang

Hack on rich terminal user interfaces by amanzini

Description

TUIs (Textual User Interface) are a big classic of our daily workflow. Many linux users 'live' in the terminal and modern implementations have a lot to offer : unicode fonts, 24 bit colors etc.

Goals

Explore the current available solution on modern languages and implement a PoC , for example a small maze generator, porting of a classic game or just display the HackWeek cute logo.
Practice some Go / Rust coding and programming patterns
Fiddle around, hack, learn, have fun
keep a development diary, practice on project documentation

Follow this link for source code repository

includes development diary

Some ideas for inspiration:

Related projects:

https://hackweek.opensuse.org/24/projects/suse-rancher-supportconfig from @eminguez

Resources

kubectl clone: Seamlessly Clone Kubernetes Resources Across Multiple Rancher Clusters and Projects by dpunia

Description

kubectl clone is a kubectl plugin that empowers users to clone Kubernetes resources across multiple clusters and projects managed by Rancher. It simplifies the process of duplicating resources from one cluster to another or within different namespaces and projects, with optional on-the-fly modifications. This tool enhances multi-cluster resource management, making it invaluable for environments where Rancher orchestrates numerous Kubernetes clusters.

Goals

Seamless Multi-Cluster Cloning
- Clone Kubernetes resources across clusters/projects with one command.
- Simplifies management, reduces operational effort.

Resources

Rancher & Kubernetes Docs
- Rancher API, Cluster Management, Kubernetes client libraries.
Development Tools
- Kubectl plugin docs, Go programming resources.

Building and Installing the Plugin

Set Environment Variables: Export the Rancher URL and API token:

export RANCHER_URL="https://rancher.example.com"
export RANCHER_TOKEN="token-xxxxx:xxxxxxxxxxxxxxxxxxxx"

Build the Plugin: Compile the Go program:

go build -o kubectl-clone ./pkg/

Install the Plugin: Move the executable to a directory in your PATH:

mv kubectl-clone /usr/local/bin/

Ensure the file is executable:

chmod +x /usr/local/bin/kubectl-clone

Verify the Plugin Installation: Test the plugin by running:

kubectl clone --help

You should see the usage information for the kubectl-clone plugin.

Usage Examples

Clone a Deployment from One Cluster to Another:

kubectl clone --source-cluster c-abc123 --type deployment --name nginx-deployment --target-cluster c-def456 --new-name nginx-deployment-clone

Clone a Service into Another Namespace and Modify Labels:

Contribute to terraform-provider-libvirt by pinvernizzi

Description

The SUSE Manager (SUMA) teams' main tool for infrastructure automation, Sumaform, largely relies on terraform-provider-libvirt. That provider is also widely used by other teams, both inside and outside SUSE.

It would be good to help the maintainers of this project and give back to the community around it, after all the amazing work that has been already done.

If you're interested in any of infrastructure automation, Terraform, virtualization, tooling development, Go (...) it is also a good chance to learn a bit about them all by putting your hands on an interesting, real-use-case and complex project.

Goals

Get more familiar with Terraform provider development and libvirt bindings in Go
Solve some issues and/or implement some features
Get in touch with the community around the project

Resources

terraform-provider-feilong by e_bischoff

Project Description

People need to test operating systems and applications on s390 platform.

Installation from scratch solutions include:

just deploy and provision manually (with the help of ftpboot script, if you are at SUSE)
use s3270 terminal emulation (used by openQA people?)
use LXC from IBM to start CP commands and analyze the results
use zPXE to do some PXE-alike booting (used by the orthos team?)
use tessia to install from scratch using autoyast
use libvirt for s390 to do some nested virtualization on some already deployed z/VM system
directly install a Linux kernel on a LPAR and use kvm + libvirt from there

Deployment from image solutions include:

use ICIC web interface (openstack in disguise, contributed by IBM)
use ICIC from the openstack terraform provider (used by Rancher QA)
use zvm_ansible to control SMAPI
connect directly to SMAPI low-level socket interface

IBM Cloud Infrastructure Center (ICIC) harnesses the Feilong API, but you can use Feilong without installing ICIC, provided you set up a "z/VM cloud connector" into one of your VMs following this schema.

What about writing a terraform Feilong provider, just like we have the terraform libvirt provider? That would allow to transparently call Feilong from your main.tf files to deploy and destroy resources on your system/z.

Goal for Hackweek 23

My final goal is to be able to easily deploy and provision VMs automatically on a z/VM system, in a way that people might enjoy even outside of SUSE.

My technical preference is to write a terraform provider plugin, as it is the approach that involves the least software components for our deployments, while remaining clean, and compatible with our existing development infrastructure.

Goals for Hackweek 24

Feilong provider works and is used internally by SUSE Manager team. Let's push it forward!

Let's add support for fiberchannel disks and multipath.

Goals for Hackweek 25

Finish support for fiberchannel disks and multipath
Fix problems with registration on hashicorp providers registry

ClusterOps - Easily install and manage your personal kubernetes cluster by andreabenini

Description

ClusterOps is a Kubernetes installer and operator designed to streamline the initial configuration and ongoing maintenance of kubernetes clusters. The focus of this project is primarily on personal or local installations. However, the goal is to expand its use to encompass all installations of Kubernetes for local development purposes.
It simplifies cluster management by automating tasks and providing just one user-friendly YAML-based configuration config.yml.

Overview

Simplified Configuration: Define your desired cluster state in a simple YAML file, and ClusterOps will handle the rest.
Automated Setup: Automates initial cluster configuration, including network settings, storage provisioning, special requirements (for example GPUs) and essential components installation.
Ongoing Maintenance: Performs routine maintenance tasks such as upgrades, security updates, and resource monitoring.
Extensibility: Easily extend functionality with custom plugins and configurations.
Self-Healing: Detects and recovers from common cluster issues, ensuring stability, idempotence and reliability. Same operation can be performed multiple times without changing the result.
Discreet: It works only on what it knows, if you are manually configuring parts of your kubernetes and this configuration does not interfere with it you can happily continue to work on several parts and use this tool only for what is needed.

Features

distribution and engine independence. Install your favorite kubernetes engine with your package manager, execute one script and you'll have a complete working environment at your disposal.
Basic config approach. One single config.yml file with configuration requirements (add/remove features): human readable, plain and simple. All fancy configs managed automatically (ingress, balancers, services, proxy, ...).
Local Builtin ContainerHub. The default installation provides a fully configured ContainerHub available locally along with the kubernetes installation. This configuration allows the user to build, upload and deploy custom container images as they were provided from external sources. Internet public sources are still available but local development can be kept in this localhost server. Builtin ClusterOps operator will be fetched from this ContainerHub registry too.
Kubernetes official dashboard installed as a plugin, others planned too (k9s for example).
Kubevirt plugin installed and properly configured. Unleash the power of classic virtualization (KVM+QEMU) on top of Kubernetes and manage your entire system from there, libvirtd and virsh libs are required.
One operator to rule them all. The installation script configures your machine automatically during installation and adds one kubernetes operator to manage your local cluster. From there the operator takes care of the cluster on your behalf.
Clean installation and removal. Just test it, when you are done just use the same program to uninstall everything without leaving configs (or pods) behind.

Planned features (Wishlist / TODOs)

Containerized Data Importer (CDI). Persistent storage management add-on for Kubernetes to provide a declarative way of building and importing Virtual Machine Disks on PVCs for

monitoring

Update my own python audio and video time-lapse and motion capture apps and publish by dmair

Project Description

Many years ago, in my own time, I wrote a Qt python application to periodically capture frames from a V4L2 video device (e.g. a webcam) and used it to create daily weather timelapse videos from windows at my home. I have maintained it at home in my own time and this year have added motion detection making it a functional video security tool but with no guarantees. I also wrote a linux audio monitoring app in python using Qt in my own time that captures live signal strength along with 24 hour history of audio signal level/range and audio spectrum. I recently added background noise filtering to the app. In due course I aim to include voice detection, currently I'm assuming via Google's public audio interface. Neither of these is a professional home security app but between them they permit a user to freely monitor video and audio data from a home in a manageable way. Both projects are on github but out-of-date with personal work, I would like to organize and update the github versions of these projects.

Goal for this Hackweek

It would probably help to migrate all the v4l2py module based video code to linuxpy.video based code and that looks like a re-write of large areas of the video code. It would also be good to remove a lot of python lint that is several years old to improve the projects with the main goal being to push the recent changes with better organized code to github. If there is enough time I'd like to take the in-line Qt QSettings persistent state code used per-app and write a python class that encapsulates the Qt QSettings class in a value_of(name)/name=value manner for shared use in projects so that persistent state can be accessed read or write anywhere within the apps using a simple interface.

Resources

I'm not specifically looking for help but welcome other input.

Saline (state deployment control and monitoring tool for SUSE Manager/Uyuni) by vizhestkov

Project Description

Saline is an addition for salt used in SUSE Manager/Uyuni aimed to provide better control and visibility for states deploymend in the large scale environments.

In current state the published version can be used only as a Prometheus exporter and missing some of the key features implemented in PoC (not published). Now it can provide metrics related to salt events and state apply process on the minions. But there is no control on this process implemented yet.

Continue with implementation of the missing features and improve the existing implementation:

authentication (need to decide how it should be/or not related to salt auth)
web service providing the control of states deployment

Goal for this Hackweek

Implement missing key features
Implement the tool for state deployment control with CLI

Resources

https://github.com/openSUSE/saline

kubernetes

kubectl clone: Seamlessly Clone Kubernetes Resources Across Multiple Rancher Clusters and Projects by dpunia

Description

Goals

Seamless Multi-Cluster Cloning
- Clone Kubernetes resources across clusters/projects with one command.
- Simplifies management, reduces operational effort.

Resources

Rancher & Kubernetes Docs
- Rancher API, Cluster Management, Kubernetes client libraries.
Development Tools
- Kubectl plugin docs, Go programming resources.

Building and Installing the Plugin

Set Environment Variables: Export the Rancher URL and API token:

export RANCHER_URL="https://rancher.example.com"
export RANCHER_TOKEN="token-xxxxx:xxxxxxxxxxxxxxxxxxxx"

Build the Plugin: Compile the Go program:

go build -o kubectl-clone ./pkg/

Install the Plugin: Move the executable to a directory in your PATH:

mv kubectl-clone /usr/local/bin/

Ensure the file is executable:

chmod +x /usr/local/bin/kubectl-clone

Verify the Plugin Installation: Test the plugin by running:

kubectl clone --help

You should see the usage information for the kubectl-clone plugin.

Usage Examples

Clone a Deployment from One Cluster to Another:

kubectl clone --source-cluster c-abc123 --type deployment --name nginx-deployment --target-cluster c-def456 --new-name nginx-deployment-clone

Clone a Service into Another Namespace and Modify Labels:

A CLI for Harvester by mohamed.belgaied

[comment]: # Harvester does not officially come with a CLI tool, the user is supposed to interact with Harvester mostly through the UI [comment]: # Though it is theoretically possible to use kubectl to interact with Harvester, the manipulation of Kubevirt YAML objects is absolutely not user friendly. [comment]: # Inspired by tools like multipass from Canonical to easily and rapidly create one of multiple VMs, I began the development of Harvester CLI. Currently, it works but Harvester CLI needs some love to be up-to-date with Harvester v1.0.2 and needs some bug fixes and improvements as well.

Project Description

Harvester CLI is a command line interface tool written in Go, designed to simplify interfacing with a Harvester cluster as a user. It is especially useful for testing purposes as you can easily and rapidly create VMs in Harvester by providing a simple command such as: harvester vm create my-vm --count 5 to create 5 VMs named my-vm-01 to my-vm-05.

Harvester CLI is functional but needs a number of improvements: up-to-date functionality with Harvester v1.0.2 (some minor issues right now), modifying the default behaviour to create an opensuse VM instead of an ubuntu VM, solve some bugs, etc.

Github Repo for Harvester CLI: https://github.com/belgaied2/harvester-cli

Done in previous Hackweeks

Create a Github actions pipeline to automatically integrate Harvester CLI to Homebrew repositories: DONE
Automatically package Harvester CLI for OpenSUSE / Redhat RPMs or DEBs: DONE

Goal for this Hackweek

The goal for this Hackweek is to bring Harvester CLI up-to-speed with latest Harvester versions (v1.3.X and v1.4.X), and improve the code quality as well as implement some simple features and bug fixes.

Some nice additions might be: * Improve handling of namespaced objects * Add features, such as network management or Load Balancer creation ? * Add more unit tests and, why not, e2e tests * Improve CI * Improve the overall code quality * Test the program and create issues for it

Issue list is here: https://github.com/belgaied2/harvester-cli/issues

Resources

The project is written in Go, and using client-go the Kubernetes Go Client libraries to communicate with the Harvester API (which is Kubernetes in fact). Welcome contributions are:

Testing it and creating issues
Documentation
Go code improvement

What you might learn

Harvester CLI might be interesting to you if you want to learn more about:

GitHub Actions
Harvester as a SUSE Product
Go programming language
Kubernetes API

ClusterOps - Easily install and manage your personal kubernetes cluster by andreabenini

Description

Overview

Simplified Configuration: Define your desired cluster state in a simple YAML file, and ClusterOps will handle the rest.
Automated Setup: Automates initial cluster configuration, including network settings, storage provisioning, special requirements (for example GPUs) and essential components installation.
Ongoing Maintenance: Performs routine maintenance tasks such as upgrades, security updates, and resource monitoring.
Extensibility: Easily extend functionality with custom plugins and configurations.
Self-Healing: Detects and recovers from common cluster issues, ensuring stability, idempotence and reliability. Same operation can be performed multiple times without changing the result.
Discreet: It works only on what it knows, if you are manually configuring parts of your kubernetes and this configuration does not interfere with it you can happily continue to work on several parts and use this tool only for what is needed.

Features

distribution and engine independence. Install your favorite kubernetes engine with your package manager, execute one script and you'll have a complete working environment at your disposal.
Basic config approach. One single config.yml file with configuration requirements (add/remove features): human readable, plain and simple. All fancy configs managed automatically (ingress, balancers, services, proxy, ...).
Local Builtin ContainerHub. The default installation provides a fully configured ContainerHub available locally along with the kubernetes installation. This configuration allows the user to build, upload and deploy custom container images as they were provided from external sources. Internet public sources are still available but local development can be kept in this localhost server. Builtin ClusterOps operator will be fetched from this ContainerHub registry too.
Kubernetes official dashboard installed as a plugin, others planned too (k9s for example).
Kubevirt plugin installed and properly configured. Unleash the power of classic virtualization (KVM+QEMU) on top of Kubernetes and manage your entire system from there, libvirtd and virsh libs are required.
One operator to rule them all. The installation script configures your machine automatically during installation and adds one kubernetes operator to manage your local cluster. From there the operator takes care of the cluster on your behalf.
Clean installation and removal. Just test it, when you are done just use the same program to uninstall everything without leaving configs (or pods) behind.

Planned features (Wishlist / TODOs)

Containerized Data Importer (CDI). Persistent storage management add-on for Kubernetes to provide a declarative way of building and importing Virtual Machine Disks on PVCs for

Integrate Backstage with Rancher Manager by nwmacd

Description

Backstage (backstage.io) is an open-source, CNCF project that allows you to create your own developer portal. There are many plugins for Backstage.

This could be a great compliment to Rancher Manager.

Goals

Learn and experiment with Backstage and look at how this could be integrated with Rancher Manager. Goal is to have some kind of integration completed in this Hack week.

Progress

Screen shot of home page at the end of Hackweek:

Day One

Got Backstage running locally, understanding configuration with HTTPs.
Got Backstage embedded in an IFRAME inside of Rancher
Added content into the software catalog (see: https://backstage.io/docs/features/techdocs/getting-started/)
Understood more about the entity model

Day Two

Connected Backstage to the Rancher local cluster and configured the Kubernetes plugin.
Created Rancher theme to make the light theme more consistent with Rancher

Days Three and Day Four

Created two backend plugins for Backstage:
1. Catalog Entity Provider - this imports users from Rancher into Backstage
2. Auth Provider - uses the proxied sign-in pattern to check the Rancher session cookie, to user that to authenticate the user with Rancher and then log them into Backstage by connecting this to the imported User entity from the catalog entity provider plugin.
With this in place, you can single-sign-on between Rancher and Backstage when it is deployed within Rancher. Note this is only when running locally for development at present

Day Five

Start to build out a production deployment for all of the above
Made some progress, but hit issues with the authentication and proxying when running proxied within Rancher, which needs further investigation

Learn enough Golang and hack on CoreDNS by jkuzilek

Description

I'm implementing a split-horizon DNS for my home Kubernetes cluster to be able to access my internal (and external) services over the local network through public domains. I managed to make a PoC with the k8s_gateway plugin for CoreDNS. However, I soon found out it responds with IPs for all Gateways assigned to HTTPRoutes, publishing public IPs as well as the internal Loadbalancer ones.

To remedy this issue, a simple filtering mechanism has to be implemented.

Goals

Learn an acceptable amount of Golang
Implement GatewayClass (and IngressClass) filtering for k8s_gateway
Deploy on homelab cluster
Profit?

Resources

EDIT: Feature mostly complete. An unfinished PR lies here. Successfully tested working on homelab cluster.

docker

Migrate from Docker to Podman by tjyrinki_suse

Description

I'd like to continue my former work on containerization of several domains on a single server by changing from Docker containers to Podman containers. That will need an OS upgrade as well as Podman is not available in that old server version.

Goals

Update OS.
Migrate from Docker to Podman.
Keep everything functional, including the existing "meanwhile done" additional Docker container that is actually being used already.
Keep everything at least as secure as currently. One of the reasons of having the containers is to isolate risks related to services open to public Internet.
Try to enable the Podman use in production.
At minimum, learn about all of these topics.
Optionally, improve Ansible side of things as well...

Resources

A search engine is one's friend. Migrating from Docker to Podman, and from docker-compose to podman-compose.