Description
The go-git library implements the git internals in pure Go, so that any Go application can handle not only Git repositories, but also lower-level primitives (e.g. packfiles, idxfiles, etc) without needing to shell out to the git binary.
The focus for this Hackweek is to fast track key improvements for the project ahead of the upstream release of Git V3, which may take place at some point next year.
Goals
- Add support for cloning SHA256 repositories.
- Decrease memory churn for very large repositories (e.g. Linux Kernel repository).
- Cut the first alpha version for
go-git/v6.
Stretch goals
- Review and update the official documentation.
- Optimise use of go-git in Fleet.
- Create RFC/example for go-git plugins to improve extensibility.
- Investigate performance bottlenecks for Blame and Status.
Resources
- https://github.com/go-git/go-git/
- https://go-git.github.io/docs/
This project is part of:
Hack Week 25
Activity
Comments
-
7 days ago by pgomes | Reply
A copy of the official Rancher Fleet examples repo has been converted to SHA256 in GitLab to enable some of the tests: https://gitlab.com/pjbgf/fleet-examples
-
6 days ago by pgomes | Reply
The performance improvements for handling large repositories were substantial so far, with a drop in memory churn of over 99% (
1.3GBto1MB) for the Linux Kernel repo. Conversely, both allocations and time per operation dropped quite considerably as well:BenchmarkPackHandlers/packfile-cache-osfs:_linux-kernel-16 1 2240495709 ns/op 1331433920 B/op 35933 allocs/op BenchmarkPackHandlers/packfile-nocache-memfs:linux-kernel-16 1 2026880001 ns/op 1331437040 B/op 35935 allocs/op BenchmarkPackHandlers/mmap-pack-scanner:linux-kernel-16 477 2513275 ns/op 1011512 B/op 1488 allocs/opPRs:
- https://github.com/go-git/go-git/pull/1776
- https://github.com/go-git/go-git-fixtures/pull/59
-
-
4 days ago by pgomes | Reply
Hack Week final report:
- Major performance improvement in memory churn and time per operation: proposed and merged.
- A new RFC for Go support has been proposed.
- The cloning of SHA256 repositories is not quite done yet, but a lot of progress has been made to it. Changes around storage, transport, delta and packfile processing are largely done, but it needs some TLC and rounding of some rough edges.
- 2 new issues around submodule functionality were reported.
Similar Projects
git-fs: file system representation of a git repository by fgonzalez
Description
This project aims to create a Linux equivalent to the git/fs concept from git9. Now, I'm aware that git provides worktrees, but they are not enough for many use cases. Having a read-only representation of the whole repository simplifies scripting by quite a bit and, most importantly, reduces disk space usage. For instance, during kernel livepatching development, we need to process and analyze the source code of hundreds of kernel versions simultaneously.This is rather painful with git-worktrees, as each kernel branch requires no less than 1G of disk space.
As for the technical details, I'll implement the file system using FUSE. The project itself should not take much time to complete, but let's see where it takes me.
I'll try to keep the same design as git9, so the file system will look something like:
/mnt/git
+-- ctl
+-- HEAD
| +-- tree
| | +--files
| | +--in
| | +--head
| |
| +-- hash
| +-- msg
| +-- parent
|
+-- branch
| |
| +-- heads
| | +-- master
| | +-- [commit files, see HEAD]
| +-- remotes
| +-- origin
| +-- master
| +-- [commit files, see HEAD]
+-- object
+-- 00051fd3f066e8c05ae7d3cf61ee363073b9535f # blob contents
+-- 00051fd3f066e8c05ae7d3cf61ee363073b9535c
+-- [tree contents, see HEAD/tree]
+-- 3f5dbc97ae6caba9928843ec65fb3089b96c9283
+-- [commit files, see HEAD]
So, if you wanted to look at the commit message of the current branch, you could simply do:
cat /mnt/git/HEAD/msg
No collaboration needed. This is a solo project.
Goals
Implement a working prototype.
Measure and improve the performance if possible. This step will be the most crucial one. User space filesystems are slower by nature.
Resources
https://docs.kernel.org/filesystems/fuse/fuse.html
Create a page with all devel:languages:perl packages and their versions by tinita
Description
Perl projects now live in git: https://src.opensuse.org/perl
It would be useful to have an easy way to check which version of which perl module is in devel:languages:perl. Also we have meta overrides and patches for various modules, and it would be good to have them at a central place, so it is easier to lookup, and we can share with other vendors.
I did some initial data dump here a while ago: https://github.com/perlpunk/cpan-meta
But I never had the time to automate this.
I can also use the data to check if there are necessary updates (currently it uses data from download.opensuse.org, so there is some delay and it depends on building).
Goals
- Have a script that updates a central repository (e.g.
https://src.opensuse.org/perl/_metadata) with metadata by looking at https://src.opensuse.org/perl/_ObsPrj (check if there are any changes from the last run) - Create a HTML page with the list of packages (use Javascript and some table library to make it easily searchable)
Resources
Results
Day 1
- First part of the code which retrieves data from https://src.opensuse.org/perl/_ObsPrj with submodules and creates a YAML and a JSON file.
- Repo: https://github.com/perlpunk/opensuse-perl-meta
- Also a first version of the HTML is live: https://perlpunk.github.io/opensuse-perl-meta/
Day 2
- HTML Page has now links to src.opensuse.org and the date of the last update, plus a short info at the top
- Code is now 100% covered by tests: https://app.codecov.io/gh/perlpunk/opensuse-perl-meta
- I used the modern perl
classfeature, which makes perl classes even nicer and shorter. See example - Tests
- I tried out the mocking feature of the modern Test2::V0 library which provides call tracking. See example
- I tried out comparing data structures with the new Test2::V0 library. It let's you compare parts of the structure with the
likefunction, which only compares the date that is mentioned in the expected data. example
Day 3
- Added various things to the table
- Dependencies column
- Show popup with info for cpanspec, patches and dependencies
- Added last date / commit to the data export.
Plan: With the added date / commit we can now daily check _ObsPrj for changes and only fetch the data for changed packages.
Day 4
Mail client with mailing list workflow support in Rust by acervesato
Description
To create a mail user interface using Rust programming language, supporting mailing list patches workflow. I know, aerc is already there, but I would like to create something simpler, without integrated protocols. Just a plain user interface that is using some crates to read and create emails which are fetched and sent via external tools.
I already know Rust, but not the async support, which is needed in this case in order to handle events inside the mail folder and to send notifications.
Goals
- simple user interface in the style of
aerc, with some vim keybindings for motions and search - automatic run of external tools (like
mbsync) for checking emails - automatic run commands for notifications
- apply patch set from ML
- tree-sitter support with styles
Resources
- ratatui: user interface (https://ratatui.rs/)
- notify: folder watcher (https://docs.rs/notify/latest/notify/)
- mail-parser: parser for emails (https://crates.io/crates/mail-parser)
- mail-builder: create emails in proper format (https://docs.rs/mail-builder/latest/mail_builder/)
- gitpatch: ML support (https://crates.io/crates/gitpatch)
- tree-sitter-rust: support for mail format (https://crates.io/crates/tree-sitter)
Play with the userfaultfd(2) system call and download on demand using HTTP Range Requests with Golang by rbranco
Description
The userfaultfd(2) is a cool system call to handle page faults in user-space. This should allow me to list the contents of an ISO or similar archive without downloading the whole thing. The userfaultfd(2) part can also be done in theory with the PROT_NONE mprotect + SIGSEGV trick, for complete Unix portability, though reportedly being slower.
Goals
- Create my own library for userfaultfd(2) in Golang.
- Create my own library for HTTP Range Requests.
- Complete portability with Unix.
- Benchmarks.
- Contribute some tests to LTP.
Resources
- https://docs.kernel.org/admin-guide/mm/userfaultfd.html
- https://www.cons.org/cracauer/cracauer-userfaultfd.html
Create a Cloud-Native policy engine with notifying capabilities to optimize resource usage by gbazzotti
Description
The goal of this project is to begin the initial phase of development of an all-in-one Cloud-Native Policy Engine that notifies resource owners when their resources infringe predetermined policies. This was inspired by a current issue in the CES-SRE Team where other solutions seemed to not exactly correspond to the needs of the specific workloads running on the Public Cloud Team space.
The initial architecture can be checked out on the Repository listed under Resources.
Among the features that will differ this project from other monitoring/notification systems:
- Pre-defined sensible policies written at the software-level, avoiding a learning curve by requiring users to write their own policies
- All-in-one functionality: logging, mailing and all other actions are not required to install any additional plugins/packages
- Easy account management, being able to parse all required configuration by a single JSON file
- Eliminate integrations by not requiring metrics to go through a data-agreggator
Goals
- Create a minimal working prototype following the workflow specified on the documentation
- Provide instructions on installation/usage
- Work on email notifying capabilities
Resources
terraform-provider-feilong by e_bischoff
Project Description
People need to test operating systems and applications on s390 platform. While this is straightforward with KVM, this is very difficult with z/VM.
IBM Cloud Infrastructure Center (ICIC) harnesses the Feilong API, but you can use Feilong without installing ICIC(see this schema).
What about writing a terraform Feilong provider, just like we have the terraform libvirt provider? That would allow to transparently call Feilong from your main.tf files to deploy and destroy resources on your z/VM system.
Goal for Hackweek 23
I would like to be able to easily deploy and provision VMs automatically on a z/VM system, in a way that people might enjoy even outside of SUSE.
My technical preference is to write a terraform provider plugin, as it is the approach that involves the least software components for our deployments, while remaining clean, and compatible with our existing development infrastructure.
Goals for Hackweek 24
Feilong provider works and is used internally by SUSE Manager team. Let's push it forward!
Let's add support for fiberchannel disks and multipath.
Goals for Hackweek 25
Modernization, maturity, and maintenance: support for SLES 16 and openTofu, new API calls, fixes...
Resources
Outcome
SUSE Health Check Tools by roseswe
SUSE HC Tools Overview
A collection of tools written in Bash or Go 1.24++ to make life easier with handling of a bunch of tar.xz balls created by supportconfig.
Background: For SUSE HC we receive a bunch of supportconfig tar balls to check them for misconfiguration, areas for improvement or future changes.
Main focus on these HC are High Availability (pacemaker), SLES itself and SAP workloads, esp. around the SUSE best practices.
Goals
- Overall improvement of the tools
- Adding new collectors
- Add support for SLES16
Resources
csv2xls* example.sh go.mod listprodids.txt sumtext* trails.go README.md csv2xls.go exceltest.go go.sum m.sh* sumtext.go vercheck.py* config.ini csvfiles/ getrpm* listprodids* rpmdate.sh* sumxls* verdriver* credtest.go example.py getrpm.go listprodids.go sccfixer.sh* sumxls.go verdriver.go
docollall.sh* extracthtml.go gethostnamectl* go.sum numastat.go cpuvul* extractcluster.go firmwarebug* gethostnamectl.go m.sh* numastattest.go cpuvul.go extracthtml* firmwarebug.go go.mod numastat* xtr_cib.sh*
$ getrpm -r pacemaker
>> Product ID: 2795 (SUSE Linux Enterprise Server for SAP Applications 15 SP7 x86_64), RPM Name:
+--------------+----------------------------+--------+--------------+--------------------+
| Package Name | Version | Arch | Release | Repository |
+--------------+----------------------------+--------+--------------+--------------------+
| pacemaker | 2.1.10+20250718.fdf796ebc8 | x86_64 | 150700.3.3.1 | sle-ha/15.7/x86_64 |
| pacemaker | 2.1.9+20250410.471584e6a2 | x86_64 | 150700.1.9 | sle-ha/15.7/x86_64 |
+--------------+----------------------------+--------+--------------+--------------------+
Total packages found: 2
A CLI for Harvester by mohamed.belgaied
Harvester does not officially come with a CLI tool, the user is supposed to interact with Harvester mostly through the UI. Though it is theoretically possible to use kubectl to interact with Harvester, the manipulation of Kubevirt YAML objects is absolutely not user friendly. Inspired by tools like multipass from Canonical to easily and rapidly create one of multiple VMs, I began the development of Harvester CLI. Currently, it works but Harvester CLI needs some love to be up-to-date with Harvester v1.0.2 and needs some bug fixes and improvements as well.
Project Description
Harvester CLI is a command line interface tool written in Go, designed to simplify interfacing with a Harvester cluster as a user. It is especially useful for testing purposes as you can easily and rapidly create VMs in Harvester by providing a simple command such as:
harvester vm create my-vm --count 5
to create 5 VMs named my-vm-01 to my-vm-05.
Harvester CLI is functional but needs a number of improvements: up-to-date functionality with Harvester v1.0.2 (some minor issues right now), modifying the default behaviour to create an opensuse VM instead of an ubuntu VM, solve some bugs, etc.
Github Repo for Harvester CLI: https://github.com/belgaied2/harvester-cli
Done in previous Hackweeks
- Create a Github actions pipeline to automatically integrate Harvester CLI to Homebrew repositories: DONE
- Automatically package Harvester CLI for OpenSUSE / Redhat RPMs or DEBs: DONE
Goal for this Hackweek
The goal for this Hackweek is to bring Harvester CLI up-to-speed with latest Harvester versions (v1.3.X and v1.4.X), and improve the code quality as well as implement some simple features and bug fixes.
Some nice additions might be: * Improve handling of namespaced objects * Add features, such as network management or Load Balancer creation ? * Add more unit tests and, why not, e2e tests * Improve CI * Improve the overall code quality * Test the program and create issues for it
Issue list is here: https://github.com/belgaied2/harvester-cli/issues
Resources
The project is written in Go, and using client-go the Kubernetes Go Client libraries to communicate with the Harvester API (which is Kubernetes in fact).
Welcome contributions are:
- Testing it and creating issues
- Documentation
- Go code improvement
What you might learn
Harvester CLI might be interesting to you if you want to learn more about:
- GitHub Actions
- Harvester as a SUSE Product
- Go programming language
- Kubernetes API
- Kubevirt API objects (Manipulating VMs and VM Configuration in Kubernetes using Kubevirt)
