Project Description

One day in the past, a relative young geek met Wezterm terminal emulator in a wood or rusty repos. At that time he was used to drive a Win10 machine for work and a Linux shiny supercar for fun and Wezterm fit on both. So after few year the geek realize now that he should gives back to Mr. Wez something. But the geek is far to be a skilled Rustacean...

Goal for this Hackweek

  • Create some openQA needle based tests for Wezterm
  • Have the chance to go through all the exhaustive Wezterm documentation and eventually contribute to it issues

Hidden goals:

  • give back to Mr. Wez for his terminal
  • learn more about the lua wezterm configuration to improve my terminal configuration
  • learn more about openQA
  • get exposed to rust, lua, perl

Resources


Results

A mini blog of all the failure and success : opensuse.hackweek.2022

Get in touch and have fun with many different technology add-emoji - Jekyll, Liquid, Markdown, bundler, gem and other funny ruby stuff to manage the Github hosted project page. - Perl - openQA (needles and many testapi)

Some tests https://github.com/mpagot/os-autoinst-distri-opensuse/tree/weztermhackweek Some needles https://github.com/mpagot/os-autoinst-needles-opensuse/tree/weztermhackweek

https://youtu.be/mQKnl3ecxOU

Looking for hackers with the skills:

lua rust perl openqa

This project is part of:

Hack Week 21

Activity

  • over 2 years ago: xpufx liked this project.
  • over 2 years ago: okurz liked this project.
  • over 2 years ago: cdywan liked this project.
  • over 2 years ago: mpagot started this project.
  • over 2 years ago: mbrugger liked this project.
  • over 2 years ago: mpagot added keyword "rust" to this project.
  • over 2 years ago: mpagot added keyword "perl" to this project.
  • over 2 years ago: mpagot added keyword "openqa" to this project.
  • over 2 years ago: mpagot added keyword "lua" to this project.
  • over 2 years ago: mpagot originated this project.

  • Comments

    • mpagot
      over 2 years ago by mpagot | Reply

      Project log on https://michelepagot.github.io/opensuse.hackweek.2022/

    Similar Projects

    A sane DSL for udev rules by mwilck

    Description

    The "language" in which udev rules are written as documented in udev(7) is horrible. To name just a few problems:

    • The only control statements are LABEL and GOTO.
    • Conditionals are the most important part of the language, but it supports only conjunction ("AND"), forcing developers to use GOTO even for simple "OR" relations.
    • The AND operation is denoted by a comma (,).
    • Conventions for quoting are weird.
    • There aren't even basic string handling facilities.
    • There is no API for passing flags between different rule sets, just a set of global environment variables.
    • Environment variales are written in ENV{FOO} when assigned to, but $env{FOO} or %E{FOO} when dereferenced.

    While this is ok-ish for the simple set of tasks the language was originally intended for, it makes larger rule sets with complex logic almost impossible to read and understand. Examples for such complex rule sets are the device-mapper and multipath rules.

    While working on the multipath rule sets a few weeks ago, I found myself desparately translating the rules into some pythonesque pseudo-code in order to make sure I fully understand the code flow.

    This project wants to explore the possibilities to replace this weird DSL with something saner. The idea is to embed Lua in udev, and rewrite the udev rule sets as Lua modules.

    It's meant as a fun project that may have practical merits. I am aware that it's questionable whether the systemd maintainers are going to embrace this. I think it will only have a tiny chance if it really improves readability of rules massively, while impacting neither performance nor code size too badly. I have good hopes in terms of performance as Lua has the reputation to be fast, but code size will of course increase, and so will the list of dependencies of systemd.

    Goals

    1. Learn how to write Lua, and how to embed it in C code and write C modules for it. I've read the manual, but I can't say that I'm fluent with it yet.
    2. Design an API for udev in Lua. Figure out how to expose status like device properties (environment variables), tags, and other variables like ACTION and SUBSYSTEM to Lua code, and tentatively re-write a set of representative rule sets in this new DSL. This may need several attempts until the rules can be written in an intuitive way.
    3. Create the basic framework for embedding Lua in udev and providing the environment for the Lua code.
    4. Write libraries with helper funtions to be used by the Lua. For example, we'll need code to access sysfs in order to provide ATTRS and similar features of udev. I'm not certain yet but for most of this it will probably make sense to write the code in C and make it callable by Lua, as udev already has powerful mechanisms for accessing sysfs attributes.
    5. One problem is that the Lua standard library provides getenv but not setenv. We need to be able to set environment variables in order to work with external programs to replicate the functionality of the PROGRAM and IMPORT{program} directives.
    6. Eventually, make the entire feature set of the udev language available.
    7. Eventually, rewrite the entire set of upstream rules (to the extent we're aware of) in Lua.
    8. While I'd love trash the original language, I guess it needs to continue to exist.


    Agama installer on-line demo by lslezak

    Description

    The Agama installer provides a quite complex user interface. We have some screenshots on the web page but as it is basically a web application it would be nice to have some on-line demo where users could click and check it live.

    The problem is that the Agama server directly accesses the hardware (storage probing) and loads installation repositories. We cannot easily mock this in the on-line demo so the easiest way is to have just a read-only demo. You could explore the configuration options but you could not change anything, all changes would be ignored.

    The read-only demo would be a bit limited but I still think it would be useful for potential users get the feeling of the new Agama installer and get familiar with it before using in a real installation.

    As a proof of concept I already created this on-line demo.

    The implementation basically builds Agama in two modes - recording mode where it saves all REST API responses and replay mode where it for the REST API requests returns the previously recorded responses. Recording in the browser is inconvenient and error prone, there should be some scripting instead (see below).

    Goals

    • Create an Agama on-line demo which can be easily tested by users
    • The Agama installer is still in alpha phase and in active development, the online demo needs to be easily rebuilt with the latest Agama version
    • Ideally there should be some automation so the demo page is rebuilt automatically without any developer interactions (once a day or week?)

    TODO

    • Use OpenAPI to get all Agama REST API endpoints, write a script which queries all the endpoints automatically and saves the collected data to a file (see this related PR).
    • Write a script for starting an Agama VM (use libvirt/qemu?), the script should ensure we always use the same virtual HW so if we need to dump the latest REST API state we get the same (or very similar data). This should ensure the demo page does not change much regarding the storage proposal etc...
    • Fix changing the product, currently it gets stuck after clicking the "Select" button.
    • Move the mocking data (the recorded REST API responses) outside the Agama sources, it's too big and will be probably often updated. To avoid messing the history keep it in a separate GitHub repository
    • Allow changing the UI language
    • Display some note (watermark) in the page so it is clear it is a read-only demo (probably with some version or build date to know how old it is)
    • Automation for building new demo page from the latest sources. There should be some check which ensures the recorded data still matches the OpenAPI specification.

    Changing the UI language

    This will be quite tricky because selecting the proper translation file is done on the server side. We would probably need to completely re-implement the logic in the browser side and adapt the server for that.

    Also some REST API responses contain translated texts (storage proposal, pattern names in software). We would need to query the respective endpoints in all supported languages and return the correct response in runtime according to the currently selected language.

    Resources


    Kanidm: A safe and modern IDM system by firstyear

    Kanidm is an IDM system written in Rust for modern systems authentication. The github repo has a detailed "getting started" on the readme.

    Kanidm Github

    In addition Kanidm has spawn a number of adjacent projects in the Rust ecosystem such as LDAP, Kerberos, Webauthn, and cryptography libraries.

    In this hack week, we'll be working on Quokca, a certificate authority that supports PKCS11/TPM storage of keys, issuance of PIV certificates, and ACME without the feature gatekeeping implemented by other CA's like smallstep.

    For anyone who wants to participate in Kanidm, we have documentation and developer guides which can help.

    I'm happy to help and share more, so please get in touch!


    SMB3 Server written entirely in Rust by dmulder

    Description

    Given the number of bugs frequently discovered in the Samba code caused by memory issues, it makes sense to re-write the smbd service purely in Rust code. Meanwhile, it would be wise to abandon backwards compatibility here with insecure protocol versions, and simply implement the SMB3 spec.

    Goals

    Get a simple server up and running and get it merged into upstream Samba (which now has Rust build support).

    Resources


    Grapesss: a physical Shamir's Secret Sharing application [ESP32-C3 + Mobile] by ecandino

    drawing

    Description

    A couple of years ago I created StegoSecretS, a small cli used to encrypt and split a secret into multiple keys, using the Shamir's Secret Sharing algorithm.

    The idea is to re-implement the project using physical devices. One device alone will be useless, but when close together they can be used to decrypt the secret.

    On a practical side the user encrypts the secret with a mobile application. The same application is used to split the secret, and load the partial keys into different micro-controllers. Another user will be able to decrypt the secret only having at least N devices close together (using the application).

    I'm planning to use a couple of ESP32-C3 I bought, and build a very simple Android mobile application.

    Goals

    • Learn about Rust and micro-controllers (ESP32-C3)
    • Learn about mobile applications (Android and Kotlin)

    Resources


    Implement a CLI tool for Trento - trentoctl by nkopliku

    Description

    Implement a trentoctl CLI for interacting with a trento installation

    Goals

    • learn rust
    • implement an initial trentoctl tool to enhance trento automation
    • have fun

    Resources

    trento rust. TUIs listed on this other hackweek project Hack on rich terminal user interfaces


    Create object oriented API for perl's YAML::XS module, with YAML 1.2 Support by tinita

    Description

    YAML::XS is a binding to libyaml and already quite old, but the most popular YAML module for perl. There are two main issues:

    • It uses global package variables to influence behaviour.
    • It didn't implement the loading of types like numbers and booleans according to the YAML spec (neither 1.1 nor 1.2).

    Goals

    Create a new interface which works object oriented. Currently YAML::XS exports a list of functions.

    • The new API will allow to create a YAML::XS object containing configuration influencing the behaviour of loading and dumping.
      • It keeps the libyaml parser and emitter structs in memory, so repeated calls can save the creation of those structs
    • It will by default implement the YAML 1.2 Core Schema, so it is compatible to other YAML processors in perl and in other languages
    • If I have time, I would like to add the merge << key feature as an option. We could then use it in openQA as a replacement for YAML::PP to be faster.

    I already created a proof of concept with a minimal functionality some weeks before this HackWeek.

    Resources


    Enhance UV openQA helper script by mdonis

    Description

    A couple months ago an UV openQA helper script was created to help/automate the searching phase inside openQA for a given MU to test. The script searches inside all our openQA job groups (qam-sle) related with a given MU and generates an output suitable to add (copy & paste) inside the update log.

    This is still a WIP and could use some enhancements.

    Goals

    • Move script from bash to python: this would be useful in case we want to include this into MTUI in the future. The script will be separate from MTUI for now. The idea is to have this as a CLI tool using the click library or something similar.
    • Add option to look for jobs in other sections inside aggregated updates: right now, when looking for regression tests under aggregated updates for a given MU, the script only looks inside the Core MU job group. This is where most of the regression tests we need are located, but some MUs have their regression tests under the YaST/Containers/Security MU job groups. We should keep the Core MU group as a default, but add an option to be able to look into other job groups under aggregated updates.
    • Remove the -a option: this option is used to indicate the update ID and is mandatory right now. This is a bit weird and goes against posix stardards. It was developed this way in order to avoid using positional parameters. This problem should be fixed if we move the script to python.

    Some other ideas to consider:

    • Look into the QAM dashboard API. This has more info on each MU, could use this to link general openQA build results, whether the related RR is approved or not, etc
    • Make it easier to see if there's regression tests for a package in an openQA test build. Check if there's a possibility to search for tests that have the package name in them inside each testsuite.
    • Unit testing?

    More ideas TBD

    Resources

    https://github.com/os-autoinst/scripts/blob/master/openqa-search-maintenance-core-jobs

    https://confluence.suse.com/display/maintenanceqa/Guide+on+how+to+test+Updates

    Post-Hackweek update

    All major features were implemented. Unit tests are still in progress, and project will be moved to the SUSE github org once everything's done. https://github.com/mjdonis/oqa-search


    OpenQA Golang api client by hilchev

    Description

    I would like to make a simple cli tool to communicate with the OpenQA API

    Goals

    • OpenQA has a ton of information that is hard to get via the UI. A tool like this would make my life easier :)
    • Would potentially make it easier in the future to make UI changes without Perl.
    • Improve my Golang skills

    Resources

    • https://go.dev/doc/
    • https://openqa.opensuse.org/api


    Learn obs/ibs sync tool by xlai

    Description

    Once images/repo are built from IBS/OBS, there is a tool to sync the image from IBS/OBS to openqa asset directory and trigger openqa jobs accordingly.

    Goals

    Check how the tool is implemented, and be capable to add/modify our needed images/repo in future by ourselves.

    Resources

    • https://github.com/os-autoinst/openqa-trigger-from-obs
    • https://gitlab.suse.de/openqa/openqa-trigger-from-ibs-plugin/-/tree/master?ref_type=heads


    New features in openqa-trigger-from-obs for openQA by jlausuch

    Description

    Implement new features in openqa-trigger-from-obs to make xml more flexible.

    Goals

    One of the features to be implemented: - Possibility to define "VERSION" and "ARCH" variables per flavor instead of global.

    Resources

    https://github.com/os-autoinst/openqa-trigger-from-obs


    Make more sense of openQA test results using AI by livdywan

    Description

    AI has the potential to help with something many of us spend a lot of time doing which is making sense of openQA logs when a job fails.

    User Story

    Allison Average has a puzzled look on their face while staring at log files that seem to make little sense. Is this a known issue, something completely new or maybe related to infrastructure changes?

    Goals

    • Leverage a chat interface to help Allison
    • Create a model from scratch based on data from openQA
    • Proof of concept for automated analysis of openQA test results

    Bonus

    • Use AI to suggest solutions to merge conflicts
      • This would need a merge conflict editor that can suggest solving the conflict
    • Use image recognition for needles

    Resources

    Timeline

    Day 1

    • Conversing with open-webui to teach me how to create a model based on openQA test results

    Day 2

    Highlights

    • I briefly tested compared models to see if they would make me more productive. Between llama, gemma and mistral there was no amazing difference in the results for my case.
    • Convincing the chat interface to produce code specific to my use case required very explicit instructions.
    • Asking for advice on how to use open-webui itself better was frustratingly unfruitful both in trivial and more advanced regards.
    • Documentation on source materials used by LLM's and tools for this purpose seems virtually non-existent - specifically if a logo can be generated based on particular licenses

    Outcomes

    • Chat interface-supported development is providing good starting points and open-webui being open source is more flexible than Gemini. Although currently some fancy features such as grounding and generated podcasts are missing.
    • Allison still has to be very experienced with openQA to use a chat interface for test review. Publicly available system prompts would make that easier, though.