Uyuni Saltboot rework by oholecek

Description

When Uyuni switched over to the containerized proxies we had to abandon salt based saltboot infrastructure we had before. Uyuni already had integration with a Cobbler provisioning server and saltboot infra was re-implemented on top of this Cobbler integration.

What was not obvious from the start was that Cobbler, having all it's features, woefully slow when dealing with saltboot size environments. We did some improvements in performance, introduced transactions, and generally tried to make this setup usable. However the underlying slowness remained.

Goals

This project is not something trying to invent new things, it is just finally implementing saltboot infrastructure directly with the Uyuni server core.

Instead of generating grub and pxelinux configurations by Cobbler for all thousands of systems and branches, we will provide a GET access point to retrieve grub or pxelinux file during the boot:

/saltboot/group/grub/$fqdn and similar for systems /saltboot/system/grub/$mac

Next we adapt our tftpd translator to query these points when asked for default or mac based config.

Lastly similar thing needs to be done on our apache server when HTTP UEFI boot is used.

Resources

---

Flaky Tests AI Finder for Uyuni and MLM Test Suites by oscar-barrios

Description

Our current Grafana dashboards provide a great overview of test suite health, including a panel for "Top failed tests." However, identifying which of these failures are due to legitimate bugs versus intermittent "flaky tests" is a manual, time-consuming process. These flaky tests erode trust in our test suites and slow down development.

This project aims to build a simple but powerful Python script that automates flaky test detection. The script will directly query our Prometheus instance for the historical data of each failed test, using the jenkins_build_test_case_failure_age metric. It will then format this data and send it to the Gemini API with a carefully crafted prompt, asking it to identify which tests show a flaky pattern.

The final output will be a clean JSON list of the most probable flaky tests, which can then be used to populate a new "Top Flaky Tests" panel in our existing Grafana test suite dashboard.

Goals

By the end of Hack Week, we aim to have a single, working Python script that:

1. Connects to Prometheus and executes a query to fetch detailed test failure history.
2. Processes the raw data into a format suitable for the Gemini API.
3. Successfully calls the Gemini API with the data and a clear prompt.
4. Parses the AI's response to extract a simple list of flaky tests.
5. Saves the list to a JSON file that can be displayed in Grafana.
6. New panel in our Dashboard listing the Flaky tests

Resources

• Jenkins Prometheus Exporter: https://github.com/uyuni-project/jenkins-exporter/
• Data Source: Our internal Prometheus server.
• Key Metric: jenkins_build_test_case_failure_age{jobname, buildid, suite, case, status, failedsince}.
• Existing Query for Reference: count by (suite) (max_over_time(jenkins_build_test_case_failure_age{status=~"FAILED|REGRESSION", jobname="$jobname"}[$__range])).
• AI Model: The Google Gemini API.
• Example about how to interact with Gemini API: https://github.com/srbarrios/FailTale/
• Visualization: Our internal Grafana Dashboard.
• Internal IaC: https://gitlab.suse.de/galaxy/infrastructure/-/tree/master/srv/salt/monitoring

Outcome

• Jenkins Flaky Test Detector: https://github.com/srbarrios/jenkins-flaky-tests-detector and its container
• IaC on MLM Team: https://gitlab.suse.de/galaxy/infrastructure/-/tree/master/srv/salt/monitoring/jenkinsflakytestsdetector?reftype=heads, https://gitlab.suse.de/galaxy/infrastructure/-/blob/master/srv/salt/monitoring/grafana/dashboards/flaky-tests.json?ref_type=heads, and others.
• Grafana Dashboard: https://grafana.mgr.suse.de/d/flaky-tests/flaky-tests-detection @ @ text

---

Ansible to Salt integration by vizhestkov

Description

We already have initial integration of Ansible in Salt with the possibility to run playbooks from the salt-master on the salt-minion used as an Ansible Control node.

In this project I want to check if it possible to make Ansible working on the transport of Salt. Basically run playbooks with Ansible through existing established Salt (ZeroMQ) transport and not using ssh at all.

It could be a good solution for the end users to reuse Ansible playbooks or run Ansible modules they got used to with no effort of complex configuration with existing Salt (or Uyuni/SUSE Multi Linux Manager) infrastructure.

Goals

• [v] Prepare the testing environment with Salt and Ansible installed
• [v] Discover Ansible codebase to figure out possible ways of integration
• [v] Create Salt/Uyuni inventory module
• [v] Make basic modules to work with no using separate ssh connection, but reusing existing Salt connection
• [v] Test some most basic playbooks

Resources

GitHub page

Video of the demo

---

Uyuni read-only replica by cbosdonnat

Description

For now, there is no possible HA setup for Uyuni. The idea is to explore setting up a read-only shadow instance of an Uyuni and make it as useful as possible.

Possible things to look at:

• live sync of the database, probably using the WAL. Some of the tables may have to be skipped or some features disabled on the RO instance (taskomatic, PXT sessions…)
• Can we use a load balancer that routes read-only queries to either instance and the other to the RW one? For example, packages or PXE data can be served by both, the API GET requests too. The rest would be RW.

Goals

• Prepare a document explaining how to do it.
• PR with the needed code changes to support it

---

Set Uyuni to manage edge clusters at scale by RDiasMateus

Description

Prepare a Poc on how to use MLM to manage edge clusters. Those cluster are normally equal across each location, and we have a large number of them.

The goal is to produce a set of sets/best practices/scripts to help users manage this kind of setup.

Goals

step 1: Manual set-up

Goal: Have a running application in k3s and be able to update it using System Update Controler (SUC)

• Deploy Micro 6.2 machine

• Deploy k3s - single node

  • https://docs.k3s.io/quick-start

• Build/find a simple web application (static page)

  • Build/find a helmchart to deploy the application

• Deploy the application on the k3s cluster

• Install App updates through helm update

• Install OS updates using MLM

step 2: Automate day 1

Goal: Trigger the application deployment and update from MLM

• Salt states For application (with static data)
  • Deploy the application helmchart, if not present
  • install app updates through helmchart parameters
• Link it to GIT
  • Define how to link the state to the machines (based in some pillar data? Using configuration channels by importing the state? Naming convention?)
  • Use git update to trigger helmchart app update
• Recurrent state applying configuration channel?

step 3: Multi-node cluster

Goal: Use SUC to update a multi-node cluster.

• Create a multi-node cluster
• Deploy application
  • call the helm update/install only on control plane?
• Install App updates through helm update
• Prepare a SUC for OS update (k3s also? How?)
  • https://github.com/rancher/system-upgrade-controller
  • https://documentation.suse.com/cloudnative/k3s/latest/en/upgrades/automated.html
  • Update/deploy the SUC?
  • Update/deploy the SUC CRD with the update procedure

---

Preparing KubeVirtBMC for project transfer to the KubeVirt organization by zchang

Description

KubeVirtBMC is preparing to transfer the project to the KubeVirt organization. One requirement is to enhance the modeling design's security. The current v1alpha1 API (the VirtualMachineBMC CRD) was designed during the proof-of-concept stage. It's immature and inherently insecure due to its cross-namespace object references, exposing security concerns from an RBAC perspective.

The other long-awaited feature is the ability to mount virtual media so that virtual machines can boot from remote ISO images.

Goals

1. Deliver the v1beta1 API and its corresponding controller implementation
2. Enable the Redfish virtual media mount function for KubeVirt virtual machines

Resources

• The KubeVirtBMC repo: https://github.com/starbops/kubevirtbmc
• The new v1beta1 API: https://github.com/starbops/kubevirtbmc/issues/83
• Redfish virtual media mount: https://github.com/starbops/kubevirtbmc/issues/44

---

Technical talks at universities by agamez

Description

This project aims to empower the next generation of tech professionals by offering hands-on workshops on containerization and Kubernetes, with a strong focus on open-source technologies. By providing practical experience with these cutting-edge tools and fostering a deep understanding of open-source principles, we aim to bridge the gap between academia and industry.

For now, the scope is limited to Spanish universities, since we already have the contacts and have started some conversations.

Goals

• Technical Skill Development: equip students with the fundamental knowledge and skills to build, deploy, and manage containerized applications using open-source tools like Kubernetes.
• Open-Source Mindset: foster a passion for open-source software, encouraging students to contribute to open-source projects and collaborate with the global developer community.
• Career Readiness: prepare students for industry-relevant roles by exposing them to real-world use cases, best practices, and open-source in companies.

Resources

• Instructors: experienced open-source professionals with deep knowledge of containerization and Kubernetes.
• SUSE Expertise: leverage SUSE's expertise in open-source technologies to provide insights into industry trends and best practices.

---

OpenPlatform Self-Service Portal by tmuntan1

Description

In SUSE IT, we developed an internal developer platform for our engineers using SUSE technologies such as RKE2, SUSE Virtualization, and Rancher. While it works well for our existing users, the onboarding process could be better.

To improve our customer experience, I would like to build a self-service portal to make it easy for people to accomplish common actions. To get started, I would have the portal create Jira SD tickets for our customers to have better information in our tickets, but eventually I want to add automation to reduce our workload.

Goals

• Build a frontend website (Angular) that helps customers create Jira SD tickets.
• Build a backend (Rust with Axum) for the backend, which would do all the hard work for the frontend.

Resources (SUSE VPN only)

• development site: https://ui-dev.openplatform.suse.com/login?returnUrl=%2Fopenplatform%2Fforms
• https://gitlab.suse.de/itpe/core/open-platform/op-portal/backend
• https://gitlab.suse.de/itpe/core/open-platform/op-portal/frontend

---

Rancher/k8s Trouble-Maker by tonyhansen

Project Description

When studying for my RHCSA, I found trouble-maker, which is a program that breaks a Linux OS and requires you to fix it. I want to create something similar for Rancher/k8s that can allow for troubleshooting an unknown environment.

Goals for Hackweek 25

• Update to modern Rancher and verify that existing tests still work
• Change testing logic to populate secrets instead of requiring a secondary script
• Add new tests

Goals for Hackweek 24 (Complete)

• Create a basic framework for creating Rancher/k8s cluster lab environments as needed for the Break/Fix
• Create at least 5 modules that can be applied to the cluster and require troubleshooting

Resources

• https://github.com/celidon/rancher-troublemaker
• https://github.com/rancher/terraform-provider-rancher2
• https://github.com/rancher/tf-rancher-up
• https://github.com/rancher/quickstart

---

Kubernetes-Based ML Lifecycle Automation by lmiranda

Description

This project aims to build a complete end-to-end Machine Learning pipeline running entirely on Kubernetes, using Go, and containerized ML components.

The pipeline will automate the lifecycle of a machine learning model, including:

• Data ingestion/collection
• Model training as a Kubernetes Job
• Model artifact storage in an S3-compatible registry (e.g. Minio)
• A Go-based deployment controller that automatically deploys new model versions to Kubernetes using Rancher
• A lightweight inference service that loads and serves the latest model
• Monitoring of model performance and service health through Prometheus/Grafana

The outcome is a working prototype of an MLOps workflow that demonstrates how AI workloads can be trained, versioned, deployed, and monitored using the Kubernetes ecosystem.

Goals

By the end of Hack Week, the project should:

1. Produce a fully functional ML pipeline running on Kubernetes with:

   • Data collection job
   • Training job container
   • Storage and versioning of trained models
   • Automated deployment of new model versions
   • Model inference API service
   • Basic monitoring dashboards

2. Showcase a Go-based deployment automation component, which scans the model registry and automatically generates & applies Kubernetes manifests for new model versions.

3. Enable continuous improvement by making the system modular and extensible (e.g., additional models, metrics, autoscaling, or drift detection can be added later).

4. Prepare a short demo explaining the end-to-end process and how new models flow through the system.

Resources

Project Repository

Updates

1. Training pipeline and datasets
2. Inference Service py

---

SUSE Health Check Tools by roseswe

SUSE HC Tools Overview

A collection of tools written in Bash or Go 1.24++ to make life easier with handling of a bunch of tar.xz balls created by supportconfig.

Background: For SUSE HC we receive a bunch of supportconfig tar balls to check them for misconfiguration, areas for improvement or future changes.

Main focus on these HC are High Availability (pacemaker), SLES itself and SAP workloads, esp. around the SUSE best practices.

Goals

• Overall improvement of the tools
• Adding new collectors
• Add support for SLES16

Resources

csv2xls* example.sh go.mod listprodids.txt sumtext* trails.go README.md csv2xls.go exceltest.go go.sum m.sh* sumtext.go vercheck.py* config.ini csvfiles/ getrpm* listprodids* rpmdate.sh* sumxls* verdriver* credtest.go example.py getrpm.go listprodids.go sccfixer.sh* sumxls.go verdriver.go

docollall.sh* extracthtml.go gethostnamectl* go.sum numastat.go cpuvul* extractcluster.go firmwarebug* gethostnamectl.go m.sh* numastattest.go cpuvul.go extracthtml* firmwarebug.go go.mod numastat* xtr_cib.sh*

$ getrpm -r pacemaker >> Product ID: 2795 (SUSE Linux Enterprise Server for SAP Applications 15 SP7 x86_64), RPM Name: +--------------+----------------------------+--------+--------------+--------------------+ | Package Name | Version | Arch | Release | Repository | +--------------+----------------------------+--------+--------------+--------------------+ | pacemaker | 2.1.10+20250718.fdf796ebc8 | x86_64 | 150700.3.3.1 | sle-ha/15.7/x86_64 | | pacemaker | 2.1.9+20250410.471584e6a2 | x86_64 | 150700.1.9 | sle-ha/15.7/x86_64 | +--------------+----------------------------+--------+--------------+--------------------+ Total packages found: 2

---

Updatecli Autodiscovery supporting WASM plugins by olblak

Description

Updatecli is a Golang Update policy engine that allow to write Update policies in YAML manifest. Updatecli already has a plugin ecosystem for common update strategies such as automating Dockerfile or Kubernetes manifest from Git repositories.

This is what we call autodiscovery where Updatecli generate manifest and apply them dynamically based on some context.

Obviously, the Updatecli project doesn't accept plugins specific to an organization.

I saw project using different languages such as python, C#, or JS to generate those manifest.

It would be great to be able to share and reuse those specific plugins

During the HackWeek, I'll hang on the Updatecli matrix channel

https://matrix.to/#/#Updatecli_community:gitter.im

Goals

Implement autodiscovery plugins using WASM. I am planning to experiment with https://github.com/extism/extism

To build a simple WASM autodiscovery plugin and run it from Updatecli

Resources

• https://github.com/extism/extism
• https://github.com/updatecli/updatecli
• https://www.updatecli.io/docs/core/autodiscovery/
• https://matrix.to/#/#Updatecli_community:gitter.im

---

Create a Cloud-Native policy engine with notifying capabilities to optimize resource usage by gbazzotti

Description

The goal of this project is to begin the initial phase of development of an all-in-one Cloud-Native Policy Engine that notifies resource owners when their resources infringe predetermined policies. This was inspired by a current issue in the CES-SRE Team where other solutions seemed to not exactly correspond to the needs of the specific workloads running on the Public Cloud Team space.

The initial architecture can be checked out on the Repository listed under Resources.

Among the features that will differ this project from other monitoring/notification systems:

• Pre-defined sensible policies written at the software-level, avoiding a learning curve by requiring users to write their own policies
• All-in-one functionality: logging, mailing and all other actions are not required to install any additional plugins/packages
• Easy account management, being able to parse all required configuration by a single JSON file
• Eliminate integrations by not requiring metrics to go through a data-agreggator

Goals

• Create a minimal working prototype following the workflow specified on the documentation
• Provide instructions on installation/usage
• Work on email notifying capabilities

Resources

• GitHub Repository

---

go-git: unlocking SHA256-based repository cloning ahead of git v3 by pgomes

Description

The go-git library implements the git internals in pure Go, so that any Go application can handle not only Git repositories, but also lower-level primitives (e.g. packfiles, idxfiles, etc) without needing to shell out to the git binary.

The focus for this Hackweek is to fast track key improvements for the project ahead of the upstream release of Git V3, which may take place at some point next year.

Goals

• Add support for cloning SHA256 repositories.
• Decrease memory churn for very large repositories (e.g. Linux Kernel repository).
• Cut the first alpha version for go-git/v6.

Stretch goals

• Review and update the official documentation.
• Optimise use of go-git in Fleet.
• Create RFC/example for go-git plugins to improve extensibility.
• Investigate performance bottlenecks for Blame and Status.

Resources

• https://github.com/go-git/go-git/
• https://go-git.github.io/docs/

---

Contribute to terraform-provider-libvirt by pinvernizzi

Description

The SUSE Manager (SUMA) teams' main tool for infrastructure automation, Sumaform, largely relies on terraform-provider-libvirt. That provider is also widely used by other teams, both inside and outside SUSE.

It would be good to help the maintainers of this project and give back to the community around it, after all the amazing work that has been already done.

If you're interested in any of infrastructure automation, Terraform, virtualization, tooling development, Go (...) it is also a good chance to learn a bit about them all by putting your hands on an interesting, real-use-case and complex project.

Goals

• Get more familiar with Terraform provider development and libvirt bindings in Go
• Solve some issues and/or implement some features
• Get in touch with the community around the project

Resources

• CONTRIBUTING readme
• Go libvirt library in use by the project
• Terraform plugin development
• "Good first issue" list

---