Mash is a Python based CI/CD pipeline for automated testing and publishing of public cloud images. Currently the production and development deployment for the package is inconsistent, slow and manual. This is a barrier to rapid development, deployment and testing. It also means the development workflow is different than production. This can lead to production issues which were not seen during development.

In order to modernize the Mash workflow I plan to spend the week digging into a plethora of tools to first learn then build out a new workflow. The goal is to simplify deployment by choosing tools that provide consistency, modularity and repeatability. By leveraging the best tools available we can harden the code and accelerate the release cycle.

Key metrics to target are:

Mash deployment and runtime environment is consistent (dev, staging, prod, etc.).
Mash runtime environment is decoupled from the host.
Mash can be deployed with one click.
Any Mash branch in GitHub can be deployed with a single config change.
Mash has an automated E2E test suite using the new deployment

Technologies to investigate:

Docker, Podman, containers
OBS (packages, images)
Salt
Terraform (stretch goal to handle cloud instance management)
Pytest, Testinfra, img-proof for E2E testing

Join this project Leave this project

Looking for hackers with the skills:

containers podman deployment iac salt terraform ci/cd python3 docker obs

This project is part of:

Hack Week 19

Activity

almost 6 years ago: jesus_bv joined this project.

almost 6 years ago: seanmarlow added keyword "obs" to this project.

almost 6 years ago: seanmarlow added keyword "docker" to this project.

almost 6 years ago: seanmarlow added keyword "containers" to this project.

almost 6 years ago: seanmarlow added keyword "podman" to this project.

almost 6 years ago: seanmarlow added keyword "deployment" to this project.

almost 6 years ago: seanmarlow added keyword "iac" to this project.

almost 6 years ago: seanmarlow added keyword "salt" to this project.

almost 6 years ago: seanmarlow added keyword "terraform" to this project.

almost 6 years ago: seanmarlow added keyword "ci/cd" to this project.

almost 6 years ago: seanmarlow added keyword "python3" to this project.

almost 6 years ago: seanmarlow started this project.

almost 6 years ago: seanmarlow originated this project.

Comments

Be the first to comment!

Similar Projects

containers

Rewrite Distrobox in go (POC) by fabriziosestito

Description

Rewriting Distrobox in Go.

Main benefits:

Easier to maintain and to test
Adapter pattern for different container backends (LXC, systemd-nspawn, etc.)

Goals

Build a minimal starting point with core commands
Keep the CLI interface compatible: existing users shouldn't notice any difference
Use a clean Go architecture with adapters for different container backends
Keep dependencies minimal and binary size small
Benchmark against the original shell script

Resources

Upstream project: https://github.com/89luca89/distrobox/
Distrobox site: https://distrobox.it/
ArchWiki: https://wiki.archlinux.org/title/Distrobox

Technical talks at universities by agamez

Description

This project aims to empower the next generation of tech professionals by offering hands-on workshops on containerization and Kubernetes, with a strong focus on open-source technologies. By providing practical experience with these cutting-edge tools and fostering a deep understanding of open-source principles, we aim to bridge the gap between academia and industry.

For now, the scope is limited to Spanish universities, since we already have the contacts and have started some conversations.

Goals

Technical Skill Development: equip students with the fundamental knowledge and skills to build, deploy, and manage containerized applications using open-source tools like Kubernetes.
Open-Source Mindset: foster a passion for open-source software, encouraging students to contribute to open-source projects and collaborate with the global developer community.
Career Readiness: prepare students for industry-relevant roles by exposing them to real-world use cases, best practices, and open-source in companies.

Resources

Instructors: experienced open-source professionals with deep knowledge of containerization and Kubernetes.
SUSE Expertise: leverage SUSE's expertise in open-source technologies to provide insights into industry trends and best practices.

deployment

terraform-provider-feilong by e_bischoff

Project Description

People need to test operating systems and applications on s390 platform.

Installation from scratch solutions include:

just deploy and provision manually (with the help of ftpboot script, if you are at SUSE)
use s3270 terminal emulation (used by openQA people?)
use LXC from IBM to start CP commands and analyze the results
use zPXE to do some PXE-alike booting (used by the orthos team?)
use tessia to install from scratch using autoyast
use libvirt for s390 to do some nested virtualization on some already deployed z/VM system
directly install a Linux kernel on a LPAR and use kvm + libvirt from there

Deployment from image solutions include:

use ICIC web interface (openstack in disguise, contributed by IBM)
use ICIC from the openstack terraform provider (used by Rancher QA)
use zvm_ansible to control SMAPI
connect directly to SMAPI low-level socket interface

IBM Cloud Infrastructure Center (ICIC) harnesses the Feilong API, but you can use Feilong without installing ICIC, provided you set up a "z/VM cloud connector" into one of your VMs following this schema.

What about writing a terraform Feilong provider, just like we have the terraform libvirt provider? That would allow to transparently call Feilong from your main.tf files to deploy and destroy resources on your system/z.

Goal for Hackweek 23

My final goal is to be able to easily deploy and provision VMs automatically on a z/VM system, in a way that people might enjoy even outside of SUSE.

My technical preference is to write a terraform provider plugin, as it is the approach that involves the least software components for our deployments, while remaining clean, and compatible with our existing development infrastructure.

Goals for Hackweek 24

Feilong provider works and is used internally by SUSE Manager team. Let's push it forward!

Let's add support for fiberchannel disks and multipath.

Possible goals for Hackweek 25

Modernization, maturity, and maintenance.

salt

Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil

Join the Gitter channel! https://gitter.im/uyuni-project/hackweek

Uyuni is a configuration and infrastructure management tool that saves you time and headaches when you have to manage and update tens, hundreds or even thousands of machines. It also manages configuration, can run audits, build image containers, monitor and much more!

Currently there are a few distributions that are completely untested on Uyuni or SUSE Manager (AFAIK) or just not tested since a long time, and could be interesting knowing how hard would be working with them and, if possible, fix whatever is broken.

For newcomers, the easiest distributions are those based on DEB or RPM packages. Distributions with other package formats are doable, but will require adapting the Python and Java code to be able to sync and analyze such packages (and if salt does not support those packages, it will need changes as well). So if you want a distribution with other packages, make sure you are comfortable handling such changes.

No developer experience? No worries! We had non-developers contributors in the past, and we are ready to help as long as you are willing to learn. If you don't want to code at all, you can also help us preparing the documentation after someone else has the initial code ready, or you could also help with testing :-)

The idea is testing Salt and Salt-ssh clients, but NOT traditional clients, which are deprecated.

To consider that a distribution has basic support, we should cover at least (points 3-6 are to be tested for both salt minions and salt ssh minions):

Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
Onboarding (salt minion from UI, salt minion from bootstrap scritp, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
Package management (install, remove, update...)
Patching
Applying any basic salt state (including a formula)
Salt remote commands
Bonus point: Java part for product identification, and monitoring enablement
Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)
Bonus point: testsuite enablement (https://github.com/uyuni-project/uyuni/tree/master/testsuite)

If something is breaking: we can try to fix it, but the main idea is research how supported it is right now. Beyond that it's up to each project member how much to hack :-)

If you don't have knowledge about some of the steps: ask the team
If you still don't know what to do: switch to another distribution and keep testing.

This card is for EVERYONE, not just developers. Seriously! We had people from other teams helping that were not developers, and added support for Debian and new SUSE Linux Enterprise and openSUSE Leap versions :-)

Pending

In progress

FUSS

FUSS is a complete GNU/Linux solution (server, client and desktop/standalone) based on Debian for managing an educational network.

https://fuss.bz.it/

Seems to be a Debian 12 derivative, so adding it could be quite easy.

[W] Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
[W] Onboarding (salt minion from UI, salt minion from bootstrap script, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator) --> Working for all 3 options (salt minion UI, salt minion bootstrap script and salt-ssh minion from the UI).
[W] Package management (install, remove, update...) --> Installing a new package works, needs to test the rest.
[I] Patching (if patch information is available, could require writing some code to parse it, but IIRC we have support for Ubuntu already). No patches detected. Do we support patches for Debian at all?
[W] Applying any basic salt state (including a formula)

terraform

terraform-provider-feilong by e_bischoff

Project Description

People need to test operating systems and applications on s390 platform.

Installation from scratch solutions include:

just deploy and provision manually (with the help of ftpboot script, if you are at SUSE)
use s3270 terminal emulation (used by openQA people?)
use LXC from IBM to start CP commands and analyze the results
use zPXE to do some PXE-alike booting (used by the orthos team?)
use tessia to install from scratch using autoyast
use libvirt for s390 to do some nested virtualization on some already deployed z/VM system
directly install a Linux kernel on a LPAR and use kvm + libvirt from there

Deployment from image solutions include:

use ICIC web interface (openstack in disguise, contributed by IBM)
use ICIC from the openstack terraform provider (used by Rancher QA)
use zvm_ansible to control SMAPI
connect directly to SMAPI low-level socket interface

Goal for Hackweek 23

My final goal is to be able to easily deploy and provision VMs automatically on a z/VM system, in a way that people might enjoy even outside of SUSE.

Goals for Hackweek 24

Feilong provider works and is used internally by SUSE Manager team. Let's push it forward!

Let's add support for fiberchannel disks and multipath.

Possible goals for Hackweek 25

Modernization, maturity, and maintenance.

Rancher/k8s Trouble-Maker by tonyhansen

Project Description

When studying for my RHCSA, I found trouble-maker, which is a program that breaks a Linux OS and requires you to fix it. I want to create something similar for Rancher/k8s that can allow for troubleshooting an unknown environment.

Goals for Hackweek 25

Update to modern Rancher and verify that existing tests still work
Change testing logic to populate secrets instead of requiring a secondary script
Add new tests

Goals for Hackweek 24 (Complete)

Create a basic framework for creating Rancher/k8s cluster lab environments as needed for the Break/Fix
Create at least 5 modules that can be applied to the cluster and require troubleshooting

Resources

https://github.com/celidon/rancher-troublemaker
https://github.com/rancher/terraform-provider-rancher2
https://github.com/rancher/tf-rancher-up
https://github.com/rancher/quickstart

Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil

Join the Gitter channel! https://gitter.im/uyuni-project/hackweek

The idea is testing Salt and Salt-ssh clients, but NOT traditional clients, which are deprecated.

To consider that a distribution has basic support, we should cover at least (points 3-6 are to be tested for both salt minions and salt ssh minions):

Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
Onboarding (salt minion from UI, salt minion from bootstrap scritp, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
Package management (install, remove, update...)
Patching
Applying any basic salt state (including a formula)
Salt remote commands
Bonus point: Java part for product identification, and monitoring enablement
Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)
Bonus point: testsuite enablement (https://github.com/uyuni-project/uyuni/tree/master/testsuite)

If something is breaking: we can try to fix it, but the main idea is research how supported it is right now. Beyond that it's up to each project member how much to hack :-)

If you don't have knowledge about some of the steps: ask the team
If you still don't know what to do: switch to another distribution and keep testing.

Pending

In progress

FUSS

FUSS is a complete GNU/Linux solution (server, client and desktop/standalone) based on Debian for managing an educational network.

https://fuss.bz.it/

Seems to be a Debian 12 derivative, so adding it could be quite easy.

[W] Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
[W] Onboarding (salt minion from UI, salt minion from bootstrap script, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator) --> Working for all 3 options (salt minion UI, salt minion bootstrap script and salt-ssh minion from the UI).
[W] Package management (install, remove, update...) --> Installing a new package works, needs to test the rest.
[I] Patching (if patch information is available, could require writing some code to parse it, but IIRC we have support for Ubuntu already). No patches detected. Do we support patches for Debian at all?
[W] Applying any basic salt state (including a formula)

python3

Improve/rework household chore tracker `chorazon` by gniebler

Description

I wrote a household chore tracker named chorazon, which is meant to be deployed as a web application in the household's local network.

It features the ability to set up different (so far only weekly) schedules per task and per person, where tasks may span several days.

There are "tokens", which can be collected by users. Tasks can (and usually will) have rewards configured where they yield a certain amount of tokens. The idea is that they can later be redeemed for (surprise) gifts, but this is not implemented yet. (So right now one needs to edit the DB manually to subtract tokens when they're redeemed.)

Days are not rolled over automatically, to allow for task completion control.

We used it in my household for several months, with mixed success. There are many limitations in the system that would warrant a revisit.

It's written using the Pyramid Python framework with URL traversal, ZODB as the data store and Web Components for the frontend.

Goals

Add admin screens for users, tasks and schedules
Add models, pages etc. to allow redeeming tokens for gifts/surprises
…?

Resources

tbd (Gitlab repo)

openQA log viewer by mpagot

Description

*** Warning: Are You at Risk for VOMIT? ***

Do you find yourself staring at a screen, your eyes glossing over as thousands of lines of text scroll by? Do you feel a wave of text-based nausea when someone asks you to "just check the logs"?

You may be suffering from VOMIT (Verbose Output Mental Irritation Toxicity).

This dangerous, work-induced ailment is triggered by exposure to an overwhelming quantity of log data, especially from parallel systems. The human brain, not designed to mentally process 12 simultaneous autoinst-log.txt files, enters a state of toxic shock. It rejects the "Verbose Output," making it impossible to find the one critical error line buried in a 50,000-line sea of "INFO: doing a thing."

Before you're forced to rm -rf /var/log in a fit of desperation, we present the digital antacid.

No panic: The openQA Log Visualizer (Also known as the "VOMIT-B-Gone 9000")

This is your web-based hazmat suit for handling toxic log environments. It bravely dives into the chaotic, multi-machine mess of your openQA test runs, finds all the related, verbose logs, and force-feeds them into a parser.

image

Goals

Work on the existing POC openqa-log-visualizer and change it to something usable

Resources

openqa-log-visualizer

Improve chore and screen time doc generator script `wochenplaner` by gniebler

Description

I wrote a little Python script to generate PDF docs, which can be used to track daily chore completion and screen time usage for several people, with one page per person/week.

I named this script wochenplaner and have been using it for a few months now.

It needs some improvements and adjustments in how the screen time should be tracked and how chores are displayed.

Goals

Fix chore field separation lines
Change screen time tracking logic from "global" (week-long) to daily subtraction and weekly addition of remainders (more intuitive than current "weekly time budget method)
Add logic to fill in chore fields/lines, ideally with pictures, falling back to text.

Resources

tbd (Gitlab repo)

obs

Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil

Join the Gitter channel! https://gitter.im/uyuni-project/hackweek

The idea is testing Salt and Salt-ssh clients, but NOT traditional clients, which are deprecated.

To consider that a distribution has basic support, we should cover at least (points 3-6 are to be tested for both salt minions and salt ssh minions):

Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
Onboarding (salt minion from UI, salt minion from bootstrap scritp, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
Package management (install, remove, update...)
Patching
Applying any basic salt state (including a formula)
Salt remote commands
Bonus point: Java part for product identification, and monitoring enablement
Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)
Bonus point: testsuite enablement (https://github.com/uyuni-project/uyuni/tree/master/testsuite)

If something is breaking: we can try to fix it, but the main idea is research how supported it is right now. Beyond that it's up to each project member how much to hack :-)

If you don't have knowledge about some of the steps: ask the team
If you still don't know what to do: switch to another distribution and keep testing.

Pending

In progress

FUSS

FUSS is a complete GNU/Linux solution (server, client and desktop/standalone) based on Debian for managing an educational network.

https://fuss.bz.it/

Seems to be a Debian 12 derivative, so adding it could be quite easy.

[W] Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
[W] Onboarding (salt minion from UI, salt minion from bootstrap script, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator) --> Working for all 3 options (salt minion UI, salt minion bootstrap script and salt-ssh minion from the UI).
[W] Package management (install, remove, update...) --> Installing a new package works, needs to test the rest.
[I] Patching (if patch information is available, could require writing some code to parse it, but IIRC we have support for Ubuntu already). No patches detected. Do we support patches for Debian at all?
[W] Applying any basic salt state (including a formula)