Preface
In a previous hackweek project (https://github.com/SUSE/hackweek/wiki/osc3%3A-port-osc-to-Python3) I have converted osc code to Python3. This has triggered quite some weirdness regarding plugins, but osc is usable from python3. Or it was, not sure if more recent changes did not break the compatibility.
The task
But usable means, you can run osc if you don't need to bother with ssl. Which you need to, so osc3 is barely usable in a real world. osc interfaces with openssl through M2Crypto, which lacks python3 version. At the same time, python3 got much better support for ssl. So task would be to evaluate the most easy way to switch osc's ssl to python3.
Validation
There were three possible ways I have had analyzed.
- Use python.ssl module, which would be the most viable approach.
- Check the existence of much better module
- Port M2Crypto to python3
Python ssl module does provide a nice HTTPSConnection classes. But it is mostly incompatible with python 2.6.x ssl module, so one would need to use M2Crypto for python2 and ssl for python3, which would means two codepaths. But the ship stopper was the fact M2Crypto allows to work with X509 certificates stored in ~/.config/osc/trusted-certs/
, where I did not find an alternative functionality in ssl module. A short attempt to write own X509 handling code using python-cffi showed me that OpenSSL is much more complex library than one would thing. And with almost no documentation or comments in source code.
I was not successful with a looking for much better ssl module. And there are none as far as I know. The most promising project is https://github.com/pyca/cryptography, but not finished and not dealing with X509 certs. An another opinion was https://github.com/alex/opentls, but it seems more like alex's learning project before contributing to cryptography.
So the last remaining option was to port M2Crypto to Python 3.
Result
I have found the related bug report https://bugzilla.osafoundation.org/show_bug.cgi?id=12853 and realized that Matěj Cepl from RedHat made an initial effort. So I have forked his repository https://github.com/mcepl/m2crypto and made some nice progress, which unfortunately means that all low-hanging fruits are now fixed.
https://github.com/mcepl/m2crypto/pull/1
What I learned
- much much much more details about Python3 internals and how it is different from Python2 (especially IO). I can recommend to take a look at http://www.slideshare.net/dabeaz/mastering-python-3-io
- some limited ability to write a code in SWIG
- much much much much more about OpenSSL and how hard is to get into, because of a complexity and lack of docs
- the fact that in a month when last python 2 interpreter release will appear (Oct 2013), there is no good and powerful OpenSSL binding available
This project is part of:
Hack Week 10
Activity
Comments
Be the first to comment!
Similar Projects
Symbol Relations by hli
Description
There are tools to build function call graphs based on parsing source code, for example, cscope
.
This project aims to achieve a similar goal by directly parsing the disasembly (i.e. objdump) of a compiled binary. The assembly code is what the CPU sees, therefore more "direct". This may be useful in certain scenarios, such as gdb/crash debugging.
Detailed description and Demos can be found in the README file:
Supports x86 for now (because my customers only use x86 machines), but support for other architectures can be added easily.
Tested with python3.6
Goals
Any comments are welcome.
Resources
https://github.com/lhb-cafe/SymbolRelations
symrellib.py: mplements the symbol relation graph and the disassembly parser
symrel_tracer*.py: implements tracing (-t option)
symrel.py: "cli parser"
Enhance UV openQA helper script by mdonis
Description
A couple months ago an UV openQA helper script was created to help/automate the searching phase inside openQA for a given MU to test. The script searches inside all our openQA job groups (qam-sle) related with a given MU and generates an output suitable to add (copy & paste) inside the update log.
This is still a WIP and could use some enhancements.
Goals
- Move script from bash to python: this would be useful in case we want to include this into MTUI in the future. The script will be separate from MTUI for now. The idea is to have this as a CLI tool using the click library or something similar.
- Add option to look for jobs in other sections inside aggregated updates: right now, when looking for regression tests under aggregated updates for a given MU, the script only looks inside the Core MU job group. This is where most of the regression tests we need are located, but some MUs have their regression tests under the YaST/Containers/Security MU job groups. We should keep the Core MU group as a default, but add an option to be able to look into other job groups under aggregated updates.
- Remove the
-a
option: this option is used to indicate the update ID and is mandatory right now. This is a bit weird and goes against posix stardards. It was developed this way in order to avoid using positional parameters. This problem should be fixed if we move the script to python.
Some other ideas to consider:
- Look into the QAM dashboard API. This has more info on each MU, could use this to link general openQA build results, whether the related RR is approved or not, etc
- Make it easier to see if there's regression tests for a package in an openQA test build. Check if there's a possibility to search for tests that have the package name in them inside each testsuite.
- Unit testing?
More ideas TBD
Resources
https://github.com/os-autoinst/scripts/blob/master/openqa-search-maintenance-core-jobs
https://confluence.suse.com/display/maintenanceqa/Guide+on+how+to+test+Updates
Post-Hackweek update
All major features were implemented. Unit tests are still in progress, and project will be moved to the SUSE github org once everything's done. https://github.com/mjdonis/oqa-search
Selenium with Python by xguo
Description
Try to create test case about Selenium base on Python
Goals
- Knowledge about Selenium with Python
- Create new test case about Selenium
Resources
https://selenium-python.readthedocs.io/ https://www.selenium.dev/
Small healthcheck tool for Longhorn by mbrookhuis
Project Description
We have often problems (e.g. pods not starting) that are related to PVCs not running, cluster (nodes) not all up or deployments not running or completely running. This all prevents administration activities. Having something that can regular be run to validate the status of the cluster would be helpful, and not as of today do a lot of manual tasks.
As addition (read enough time), we could add changing reservation, adding new disks, etc. --> This didn't made it. But the scripts can easily be adopted.
This tool would decrease troubleshooting time, giving admins rights to the rancher GUI and could be used in automation.
Goal for this Hackweek
At the end we should have a small python tool that is doing a (very) basic health check on nodes, deployments and PVCs. First attempt was to make it in golang, but that was taking to much time.
Overview
This tool will run a simple healthcheck on a kubernetes cluster. It will perform the following actions:
node check: This will check all nodes, and display the status and the k3s version. If the status of the nodes is not "Ready" (this should be only reported), the cluster will be reported as having problems
deployment check: This check will list all deployments, and display the number of expected replicas and the used replica. If there are unused replicas this will be displayed. The cluster will be reported as having problems.
pvc check: This check will list of all pvc's, and display the status and the robustness. If the robustness is not "Healthy", the cluster will be reported as having problems.
If there is a problem registered in the checks, there will be a warning that the cluster is not healthy and the program will exit with 1.
The script has 1 mandatory parameter and that is the kubeconf of the cluster or of a node off the cluster.
The code is writen for Python 3.11, but will also work on 3.6 (the default with SLES15.x). There is a venv present that will contain all needed packages. Also, the script can be run on the cluster itself or any other linux server.
Installation
To install this project, perform the following steps:
- Create the directory /opt/k8s-check
mkdir /opt/k8s-check
- Copy all the file to this directory and make the following changes:
chmod +x k8s-check.py
Saline (state deployment control and monitoring tool for SUSE Manager/Uyuni) by vizhestkov
Project Description
Saline is an addition for salt used in SUSE Manager/Uyuni aimed to provide better control and visibility for states deploymend in the large scale environments.
In current state the published version can be used only as a Prometheus exporter and missing some of the key features implemented in PoC (not published). Now it can provide metrics related to salt events and state apply process on the minions. But there is no control on this process implemented yet.
Continue with implementation of the missing features and improve the existing implementation:
authentication (need to decide how it should be/or not related to salt auth)
web service providing the control of states deployment
Goal for this Hackweek
Implement missing key features
Implement the tool for state deployment control with CLI
Resources
https://github.com/openSUSE/saline
Symbol Relations by hli
Description
There are tools to build function call graphs based on parsing source code, for example, cscope
.
This project aims to achieve a similar goal by directly parsing the disasembly (i.e. objdump) of a compiled binary. The assembly code is what the CPU sees, therefore more "direct". This may be useful in certain scenarios, such as gdb/crash debugging.
Detailed description and Demos can be found in the README file:
Supports x86 for now (because my customers only use x86 machines), but support for other architectures can be added easily.
Tested with python3.6
Goals
Any comments are welcome.
Resources
https://github.com/lhb-cafe/SymbolRelations
symrellib.py: mplements the symbol relation graph and the disassembly parser
symrel_tracer*.py: implements tracing (-t option)
symrel.py: "cli parser"
Run local LLMs with Ollama and explore possible integrations with Uyuni by PSuarezHernandez
Description
Using Ollama you can easily run different LLM models in your local computer. This project is about exploring Ollama, testing different LLMs and try to fine tune them. Also, explore potential ways of integration with Uyuni.
Goals
- Explore Ollama
- Test different models
- Fine tuning
- Explore possible integration in Uyuni
Resources
- https://ollama.com/
- https://huggingface.co/
- https://apeatling.com/articles/part-2-building-your-training-data-for-fine-tuning/
Ansible for add-on management by lmanfredi
Description
Machines can contains various combinations of add-ons and are often modified during the time.
The list of repos can change so I would like to create an automation able to reset the status to a given state, based on metadata available for these machines
Goals
Create an Ansible automation able to take care of add-on (repo list) configuration using metadata as reference
Resources
- Machines
- Repositories
- Developing modules
- Basic VM Guest management
- Module
zypper_repository_list
- ansible-collections community.general
Results
Created WIP project Ansible-add-on-openSUSE
Team Hedgehogs' Data Observability Dashboard by gsamardzhiev
Description
This project aims to develop a comprehensive Data Observability Dashboard that provides r insights into key aspects of data quality and reliability. The dashboard will track:
Data Freshness: Monitor when data was last updated and flag potential delays.
Data Volume: Track table row counts to detect unexpected surges or drops in data.
Data Distribution: Analyze data for null values, outliers, and anomalies to ensure accuracy.
Data Schema: Track schema changes over time to prevent breaking changes.
The dashboard's aim is to support historical tracking to support proactive data management and enhance data trust across the data function.
Goals
Although the final goal is to create a power bi dashboard that we are able to monitor, our goals is to 1. Create the necessary tables that track the relevant metadata about our current data 2. Automate the process so it runs in a timely manner
Resources
AWS Redshift; AWS Glue, Airflow, Python, SQL
Why Hedgehogs?
Because we like them.
ESETv2 Emulator / interpreter by m.crivellari
Description
ESETv2 is an intriguing challenge developed by ESET, available on their website under the "Challenge" menu.
The challenge involves an "assembly-like" language and a Python compiler that generates .evm
binary files.
This is an example using one of their samples (it prints N Fibonacci numbers):
.dataSize 0
.code
loadConst 0, r1 # first
loadConst 1, r2 # second
loadConst 1, r14 # loop helper
consoleRead r3
loop:
jumpEqual end, r3, r15
add r1, r2, r4
mov r2, r1
mov r4, r2
consoleWrite r1
sub r3, r14, r3
jump loop
end:
hlt
This language also supports multi-threading. It includes instructions such as createThread
to start a new thread, joinThread
to wait until a thread completes, and lock
/unlock
to facilitate synchronization between threads.
Goals
- create a full interpreter able to run all the available samples provided by ESET.
- improve / optimize memory (eg. using bitfields where needed as well as avoid unnecessary memory allocations)
Resources
- Challenge URL: https://join.eset.com/en/challenges/core-software-engineer
- My github project: https://github.com/DispatchCode/eset_vm2 (not 100% complete)
Achivements
Project still not complete. Added lock / unlock instruction implementation but further debug is needed; there is a bug somewhere. Actually the code it works for almost all the examples in the samples folder. 1 of them is not yet runnable (due to a missing "write" opcode implementation), another will cause the bug to show up; still not investigated, anyhow.
FastFileCheck work by pstivanin
Description
FastFileCheck is a high-performance, multithreaded file integrity checker for Linux. Designed for speed and efficiency, it utilizes parallel processing and a lightweight database to quickly hash and verify large volumes of files, ensuring their integrity over time.
https://github.com/paolostivanin/FastFileCheck
Goals
- Release v1.0.0
Design overwiew:
- Main thread (producer): traverses directories and feeds the queue (one thread is more than enough for most use cases)
- Dedicated consumer thread: manages queue and distributes work to threadpool
- Worker threads: compute hashes in parallel
This separation of concerns is efficient because:
- Directory traversal is I/O bound and works well in a single thread
- Queue management is centralized, preventing race conditions
- Hash computation is CPU-intensive and properly parallelized
Add a machine-readable output to dmidecode by jdelvare
Description
There have been repeated requests for a machine-friendly dmidecode output over the last decade. During Hack Week 19, 5 years ago, I prepared the code to support alternative output formats, but didn't have the time to go further. Last year, Jiri Hnidek from Red Hat Linux posted a proof-of-concept implementation to add JSON output support. This is a fairly large pull request which needs to be carefully reviewed and tested.
Goals
Review Jiri's work and provide constructive feedback. Merge the code if acceptable. Evaluate the costs and benefits of using a library such as json-c.
FizzBuzz OS by mssola
Project Description
FizzBuzz OS (or just fbos
) is an idea I've had in order to better grasp the fundamentals of the low level of a RISC-V machine. In practice, I'd like to build a small Operating System kernel that is able to launch three processes: one that simply prints "Fizz", another that prints "Buzz", and the third which prints "FizzBuzz". These processes are unaware of each other and it's up to the kernel to schedule them by using the timer interrupts as given on openSBI (fizz on % 3 seconds, buzz on % 5 seconds, and fizzbuzz on % 15 seconds).
This kernel provides just one system call, write
, which allows any program to pass the string to be written into stdout.
This project is free software and you can find it here.
Goal for this Hackweek
- Better understand the RISC-V SBI interface.
- Better understand RISC-V in privileged mode.
- Have fun.
Resources
Results
The project was a resounding success Lots of learning, and the initial target was met.