There is a number of possible improvements to the architecture of SUSE Manager / Salt integration that should be investigated in order to improve the reliability and scalability of the backend:

Actions are currently scheduled in the minions using the schedule module of Salt. This brings problems with reliability as for instance a minion can be down at the specified schedule time which leads to actions not being executed. Scalability can be an issue as actions being scheduled for many minions might return results to the server at the same time. Instead it might be better to keep control over scheduled actions on the server to allow batching of actions as well as downtimes of minions or even the server. There is a work in progress branch to get started.
For receiving action results we are currently relying on a websocket connection to the Salt event bus in order to receive job return events. This is problematic as the connection might be interrupted leading to the server missing events. Instead we could make use of a master-side returner to write the action results directly into the postgresql database. This would further allow to have setups with multiple Salt masters returning job results to the same database.

Join this project Leave this project

Looking for hackers with the skills:

susemanager salt backend java python postgresql

This project is part of:

Hack Week 15

Activity

almost 9 years ago: dvosburg liked this project.

almost 9 years ago: whdu liked this project.

almost 9 years ago: moio joined this project.

almost 9 years ago: moio liked this project.

almost 9 years ago: j_renner added keyword "python" to this project.

almost 9 years ago: j_renner added keyword "postgresql" to this project.

almost 9 years ago: dmacvicar liked this project.

almost 9 years ago: j_renner started this project.

almost 9 years ago: j_renner liked this project.

almost 9 years ago: j_renner added keyword "susemanager" to this project.

almost 9 years ago: j_renner added keyword "salt" to this project.

almost 9 years ago: j_renner added keyword "backend" to this project.

almost 9 years ago: j_renner added keyword "java" to this project.

almost 9 years ago: j_renner originated this project.

Comments

Be the first to comment!

Similar Projects

susemanager

Set Uyuni to manage edge clusters at scale by RDiasMateus

Description

Prepare a Poc on how to use MLM to manage edge clusters. Those cluster are normally equal across each location, and we have a large number of them.

The goal is to produce a set of sets/best practices/scripts to help users manage this kind of setup.

Goals

step 1: Manual set-up

Goal: Have a running application in k3s and be able to update it using System Update Controler (SUC)

Deploy Micro 6.2 machine
Deploy k3s - single node
- https://docs.k3s.io/quick-start
Build/find a simple web application (static page)
- Build/find a helmchart to deploy the application
Deploy the application on the k3s cluster
Install App updates through helm update
Install OS updates using MLM

step 2: Multi-node cluster

Goal: Use SUC to update a multi-node cluster.

Create a multi-node cluster
Deploy application
Install App updates through helm update
Prepare a SUC for OS update (k3s also? How?)
- https://github.com/rancher/system-upgrade-controller
- https://documentation.suse.com/cloudnative/k3s/latest/en/upgrades/automated.html

step 3: Automate day 2

Goal: Trigger the application deployment and update from MLM

Salt states For application (with static data)
- Deploy the application helmchart, if not present
- Update/deploy the SUC?
- Update/deploy the SUC CRD with the update procedure
Salt states to deploy k3s cluster?
Link it to GIT
- Define how to link the state to the machines (based in some pillar data? Using configuration channels by importing the state? Naming convention?)
- Use git update to trigger helmchart app update
- Update git SUC - CR and apply the state to trigger the update of the machine.
Recurrent state applying configuration channel?

Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil

Join the Gitter channel! https://gitter.im/uyuni-project/hackweek

Uyuni is a configuration and infrastructure management tool that saves you time and headaches when you have to manage and update tens, hundreds or even thousands of machines. It also manages configuration, can run audits, build image containers, monitor and much more!

Currently there are a few distributions that are completely untested on Uyuni or SUSE Manager (AFAIK) or just not tested since a long time, and could be interesting knowing how hard would be working with them and, if possible, fix whatever is broken.

For newcomers, the easiest distributions are those based on DEB or RPM packages. Distributions with other package formats are doable, but will require adapting the Python and Java code to be able to sync and analyze such packages (and if salt does not support those packages, it will need changes as well). So if you want a distribution with other packages, make sure you are comfortable handling such changes.

No developer experience? No worries! We had non-developers contributors in the past, and we are ready to help as long as you are willing to learn. If you don't want to code at all, you can also help us preparing the documentation after someone else has the initial code ready, or you could also help with testing :-)

The idea is testing Salt and Salt-ssh clients, but NOT traditional clients, which are deprecated.

To consider that a distribution has basic support, we should cover at least (points 3-6 are to be tested for both salt minions and salt ssh minions):

Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
Onboarding (salt minion from UI, salt minion from bootstrap scritp, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
Package management (install, remove, update...)
Patching
Applying any basic salt state (including a formula)
Salt remote commands
Bonus point: Java part for product identification, and monitoring enablement
Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)
Bonus point: testsuite enablement (https://github.com/uyuni-project/uyuni/tree/master/testsuite)

If something is breaking: we can try to fix it, but the main idea is research how supported it is right now. Beyond that it's up to each project member how much to hack :-)

If you don't have knowledge about some of the steps: ask the team
If you still don't know what to do: switch to another distribution and keep testing.

This card is for EVERYONE, not just developers. Seriously! We had people from other teams helping that were not developers, and added support for Debian and new SUSE Linux Enterprise and openSUSE Leap versions :-)

Pending

Debian 13

The new version of the beloved Debian GNU/Linux OS

Seems to be a Debian 12 derivative, so adding it could be quite easy.

[ ] Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
W] Onboarding (salt minion from UI, salt minion from bootstrap script, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
[ ] Package management (install, remove, update...)
[ ] Patching (if patch information is available, could require writing some code to parse it, but IIRC we have support for Ubuntu already). Probably not for Debian as IIRC we don't support patches yet.
[ ] Applying any basic salt state (including a formula)
[ ] Salt remote commands
[ ] Bonus point: Java part for product identification, and monitoring enablement
[ ] Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
[ ] Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)

salt

Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil

Join the Gitter channel! https://gitter.im/uyuni-project/hackweek

The idea is testing Salt and Salt-ssh clients, but NOT traditional clients, which are deprecated.

To consider that a distribution has basic support, we should cover at least (points 3-6 are to be tested for both salt minions and salt ssh minions):

Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
Onboarding (salt minion from UI, salt minion from bootstrap scritp, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
Package management (install, remove, update...)
Patching
Applying any basic salt state (including a formula)
Salt remote commands
Bonus point: Java part for product identification, and monitoring enablement
Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)
Bonus point: testsuite enablement (https://github.com/uyuni-project/uyuni/tree/master/testsuite)

If something is breaking: we can try to fix it, but the main idea is research how supported it is right now. Beyond that it's up to each project member how much to hack :-)

If you don't have knowledge about some of the steps: ask the team
If you still don't know what to do: switch to another distribution and keep testing.

Pending

Debian 13

The new version of the beloved Debian GNU/Linux OS

Seems to be a Debian 12 derivative, so adding it could be quite easy.

[ ] Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
W] Onboarding (salt minion from UI, salt minion from bootstrap script, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
[ ] Package management (install, remove, update...)
[ ] Patching (if patch information is available, could require writing some code to parse it, but IIRC we have support for Ubuntu already). Probably not for Debian as IIRC we don't support patches yet.
[ ] Applying any basic salt state (including a formula)
[ ] Salt remote commands
[ ] Bonus point: Java part for product identification, and monitoring enablement
[ ] Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
[ ] Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)

backend

openQA log viewer by mpagot

Description

*** Warning: Are You at Risk for VOMIT? ***

Do you find yourself staring at a screen, your eyes glossing over as thousands of lines of text scroll by? Do you feel a wave of text-based nausea when someone asks you to "just check the logs"?

You may be suffering from VOMIT (Verbose Output Mental Irritation Toxicity).

This dangerous, work-induced ailment is triggered by exposure to an overwhelming quantity of log data, especially from parallel systems. The human brain, not designed to mentally process 12 simultaneous autoinst-log.txt files, enters a state of toxic shock. It rejects the "Verbose Output," making it impossible to find the one critical error line buried in a 50,000-line sea of "INFO: doing a thing."

Before you're forced to rm -rf /var/log in a fit of desperation, we present the digital antacid.

No panic: we have The openQA Log Visualizer

This is the UI antidote for handling toxic log environments. It bravely dives into the chaotic, multi-machine mess of your openQA test runs, finds all the related, verbose logs, and force-feeds them into a parser.

image

Goals

Work on the existing POC openqa-log-visualizer about few specific tasks:

add support for more type of logs
extend the configuration file syntax beyond the actual one
work on log parsing performance

Find some beta-tester and collect feedback and ideas about features

If time allow for it evaluate other UI frameworks and solutions (something more simple to distribute and run, maybe more low level to gain in performance).

Resources

openqa-log-visualizer

java

Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil

Join the Gitter channel! https://gitter.im/uyuni-project/hackweek

The idea is testing Salt and Salt-ssh clients, but NOT traditional clients, which are deprecated.

To consider that a distribution has basic support, we should cover at least (points 3-6 are to be tested for both salt minions and salt ssh minions):

Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
Onboarding (salt minion from UI, salt minion from bootstrap scritp, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
Package management (install, remove, update...)
Patching
Applying any basic salt state (including a formula)
Salt remote commands
Bonus point: Java part for product identification, and monitoring enablement
Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)
Bonus point: testsuite enablement (https://github.com/uyuni-project/uyuni/tree/master/testsuite)

If something is breaking: we can try to fix it, but the main idea is research how supported it is right now. Beyond that it's up to each project member how much to hack :-)

If you don't have knowledge about some of the steps: ask the team
If you still don't know what to do: switch to another distribution and keep testing.

Pending

Debian 13

The new version of the beloved Debian GNU/Linux OS

Seems to be a Debian 12 derivative, so adding it could be quite easy.

[ ] Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
W] Onboarding (salt minion from UI, salt minion from bootstrap script, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
[ ] Package management (install, remove, update...)
[ ] Patching (if patch information is available, could require writing some code to parse it, but IIRC we have support for Ubuntu already). Probably not for Debian as IIRC we don't support patches yet.
[ ] Applying any basic salt state (including a formula)
[ ] Salt remote commands
[ ] Bonus point: Java part for product identification, and monitoring enablement
[ ] Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
[ ] Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)

python

Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil

Join the Gitter channel! https://gitter.im/uyuni-project/hackweek

The idea is testing Salt and Salt-ssh clients, but NOT traditional clients, which are deprecated.

To consider that a distribution has basic support, we should cover at least (points 3-6 are to be tested for both salt minions and salt ssh minions):

Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
Onboarding (salt minion from UI, salt minion from bootstrap scritp, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
Package management (install, remove, update...)
Patching
Applying any basic salt state (including a formula)
Salt remote commands
Bonus point: Java part for product identification, and monitoring enablement
Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)
Bonus point: testsuite enablement (https://github.com/uyuni-project/uyuni/tree/master/testsuite)

If something is breaking: we can try to fix it, but the main idea is research how supported it is right now. Beyond that it's up to each project member how much to hack :-)

If you don't have knowledge about some of the steps: ask the team
If you still don't know what to do: switch to another distribution and keep testing.

Pending

Debian 13

The new version of the beloved Debian GNU/Linux OS

Seems to be a Debian 12 derivative, so adding it could be quite easy.

[ ] Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
W] Onboarding (salt minion from UI, salt minion from bootstrap script, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
[ ] Package management (install, remove, update...)
[ ] Patching (if patch information is available, could require writing some code to parse it, but IIRC we have support for Ubuntu already). Probably not for Debian as IIRC we don't support patches yet.
[ ] Applying any basic salt state (including a formula)
[ ] Salt remote commands
[ ] Bonus point: Java part for product identification, and monitoring enablement
[ ] Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
[ ] Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)

Help Create A Chat Control Resistant Turnkey Chatmail/Deltachat Relay Stack - Rootless Podman Compose, OpenSUSE BCI, Hardened, & SELinux by 3nd5h1771fy

Description

The Mission: Decentralized & Sovereign Messaging

FYI: If you have never heard of "Chatmail", you can visit their site here, but simply put it can be thought of as the underlying protocol/platform decentralized messengers like DeltaChat use for their communications. Do not confuse it with the honeypot looking non-opensource paid for prodect with better seo that directs you to chatmailsecure(dot)com

In an era of increasing centralized surveillance by unaccountable bad actors (aka BigTech), "Chat Control," and the erosion of digital privacy, the need for sovereign communication infrastructure is critical. Chatmail is a pioneering initiative that bridges the gap between classic email and modern instant messaging, offering metadata-minimized, end-to-end encrypted (E2EE) communication that is interoperable and open.

However, unless you are a seasoned sysadmin, the current recommended deployment method of a Chatmail relay is rigid, fragile, difficult to properly secure, and effectively takes over the entire host the "relay" is deployed on.

Why This Matters

A simple, host agnostic, reproducible deployment lowers the entry cost for anyone wanting to run a privacy‑preserving, decentralized messaging relay. In an era of perpetually resurrected chat‑control legislation threats, EU digital‑sovereignty drives, and many dangers of using big‑tech messaging platforms (Apple iMessage, WhatsApp, FB Messenger, Instagram, SMS, Google Messages, etc...) for any type of communication, providing an easy‑to‑use alternative empowers:

Censorship resistance - No single entity controls the relay; operators can spin up new nodes quickly.
Surveillance mitigation - End‑to‑end OpenPGP encryption ensures relay operators never see plaintext.
Digital sovereignty - Communities can host their own infrastructure under local jurisdiction, aligning with national data‑policy goals.

By turning the Chatmail relay into a plug‑and‑play container stack, we enable broader adoption, foster a resilient messaging fabric, and give developers, activists, and hobbyists a concrete tool to defend privacy online.

Goals

As I indicated earlier, this project aims to drastically simplify the deployment of Chatmail relay. By converting this architecture into a portable, containerized stack using Podman and OpenSUSE base container images, we can allow anyone to deploy their own censorship-resistant, privacy-preserving communications node in minutes.

Our goal for Hack Week: package every component into containers built on openSUSE/MicroOS base images, initially orchestrated with a single container-compose.yml (podman-compose compatible). The stack will:

Run on any host that supports Podman (including optimizations and enhancements for SELinux‑enabled systems).
Allow network decoupling by refactoring configurations to move from file-system constrained Unix sockets to internal TCP networking, allowing containers achieve stricter isolation.
Utilize Enhanced Security with SELinux by using purpose built utilities such as udica we can quickly generate custom SELinux policies for the container stack, ensuring strict confinement superior to standard/typical Docker deployments.
Allow the use of bind or remote mounted volumes for shared data (/var/vmail, DKIM keys, TLS certs, etc.).
Replace the local DNS server requirement with a remote DNS‑provider API for DKIM/TXT record publishing.

By delivering a turnkey, host agnostic, reproducible deployment, we lower the barrier for individuals and small communities to launch their own chatmail relays, fostering a decentralized, censorship‑resistant messaging ecosystem that can serve DeltaChat users and/or future services adopting this protocol

Resources

Improve chore and screen time doc generator script `wochenplaner` by gniebler

Description

I wrote a little Python script to generate PDF docs, which can be used to track daily chore completion and screen time usage for several people, with one page per person/week.

I named this script wochenplaner and have been using it for a few months now.

It needs some improvements and adjustments in how the screen time should be tracked and how chores are displayed.

Goals

Fix chore field separation lines
Change screen time tracking logic from "global" (week-long) to daily subtraction and weekly addition of remainders (more intuitive than current "weekly time budget method)
Add logic to fill in chore fields/lines, ideally with pictures, falling back to text.

Resources

tbd (Gitlab repo)

Song Search with CLAP by gcolangiuli

Description

Contrastive Language-Audio Pretraining (CLAP) is an open-source library that enables the training of a neural network on both Audio and Text descriptions, making it possible to search for Audio using a Text input. Several pre-trained models for song search are already available on huggingface

Goals

Evaluate how CLAP can be used for song searching and determine which types of queries yield the best results by developing a Minimum Viable Product (MVP) in Python. Based on the results of this MVP, future steps could include:

Music Tagging;
Free text search;
Integration with an LLM (for example, with MCP or the OpenAI API) for music suggestions based on your own library.

The code for this project will be entirely written using AI to better explore and demonstrate AI capabilities.

Resources

CLAP: The main model being researched;
huggingface: Pre-trained models for CLAP;
Free Music Archive: Creative Commons songs that can be used for testing;
Colab: To be used as the development environment;
hw25-song-search: Github repo of the project.

HTTP API for nftables by crameleon

Background

The idea originated in https://progress.opensuse.org/issues/164060 and is about building RESTful API which translates authorized HTTP requests to operations in nftables, possibly utilizing libnftables-json(5).

Originally, I started developing such an interface in Go, utilizing https://github.com/google/nftables. The conversion of string networks to nftables set elements was problematic (unfortunately no record of details), and I started a second attempt in Python, which made interaction much simpler thanks to native nftables Python bindings.

Goals

Find and track the issue with google/nftables
Revisit and polish the Python code, primarily the server component
Finish functionality to interact with nftables sets (retrieving and updating elements), which are of interest for the originating issue
Align test suite
Packaging

Resources

https://git.netfilter.org/nftables/tree/py/src/nftables.py
https://git.com.de/Georg/nftables-http-api (to be moved to GitHub)
https://build.opensuse.org/package/show/home:crameleon:containers/pytest-nftables-container

postgresql

Uyuni read-only replica by cbosdonnat

Description

For now, there is no possible HA setup for Uyuni. The idea is to explore setting up a read-only shadow instance of an Uyuni and make it as useful as possible.

Possible things to look at:

live sync of the database, probably using the WAL. Some of the tables may have to be skipped or some features disabled on the RO instance (taskomatic, PXT sessions…)
Can we use a load balancer that routes read-only queries to either instance and the other to the RW one? For example, packages or PXE data can be served by both, the API GET requests too. The rest would be RW.

Goals

Prepare a document explaining how to do it.
PR with the needed code changes to support it