Investigate possibilities for the distributed builds for Ceph to speed up builds.
This task could have 2 scopes.
replace build and
vstartfor developers to run in containers for any target base (openSUSE, Ubuntu and etc)distribute build jobs across nodes, probably on k8s cluster
There are some projects already to distribute the build:
As the first step maybe get to know already existent system and how much they speed up the build to have a reference point.
Idea is to build container (and reuse it later) with all dependencies (./install-deps.sh on some base for devs, or chroot before build phase for osc/obs) and start number of build jobs on k8s cluster (or podman if local dev env).
That could help: * developers to build, test their changes on any dist base, locally or on k8s * speed up builds for IBS/OBS to distribute build on k8s cluster
For dev environment buildah could be used to get base OS and actually run ./install-deps.sh and get current base for the build.
For osc/isc, that tool already prepares chroot, so that might be consumed as container base.
Some tool needs to be used/developed to generate k8s manifests to run build based on that container, run those manifests locally with podman play kube or remotely scheduling those manifest on k8s cluster and some how gather the results from the jobs and store resulted binaries/rpms/containers somewhere to run/store them.
This project is part of:
Hack Week 19
Activity
Comments
-
almost 6 years ago by rgrigorev | Reply
I did some measurements on our node: Main build node + icecream manage ses-client-6 main storage device /dev/nvme0n1p3 Drive speed: 'Timing buffered disk reads: 5388 MB in 3.00 seconds = 1795.90 MB/sec' CPU Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz/ 16Threads/ https://ark.intel.com/content/www/us/en/ark/products/123547/intel-xeon-silver-4110-processor-11m-cache-2-10-ghz.html 64G RAM minion nodes ses-client-7 and ses-client-8
description real user sys 1 node 210m38.879s 202m4.993s 12m45.631s 1 node -j 6 47m44.742s 254m36.362s 15m27.561s 1 node -j 11 34m38.795s 318m48.671s 17m22.380s 2 nodes -j 12 26m44.379s 156m59.053s 11m44.222s 3 nodes -j 18 25m4.973s 63m48.105s 7m53.588s 3 nodes -j 33 17m27.299s 55m18.695s 7m38.526s 3 nodes -j 33, 25G network 17m0.235s 46m24.425s 7m13.615s .... + ram disk 17m2.661s 47m0.784s 7m11.908s .... + fix in boost 13m22.895s 45m16.003s 4m55.285s
-
almost 6 years ago by rgrigorev | Reply
I did some measurements on our node:
main build node + icecream manage ses-client-6
main storage device /dev/nvme0n1p3
Drive speed: 'Timing buffered disk reads: 5388 MB in 3.00 seconds = 1795.90 MB/sec'
CPU Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz/ 16Threads/ full cpu description
64G RAM
minion nodes ses-client-7 and ses-client-8>
description ________ real __________ user __________ sys
> 1 node ______________ 210m38.879s __ 202m4.993s ___ 12m45.631s
> 1 node -j 6 ___________ 47m44.742s ___ 254m36.362s __ 15m27.561s
>1 node -j 11 __________ 34m38.795s ___ 318m48.671s __ 17m22.380s
>2 nodes -j 12 _________ 26m44.379s ___ 156m59.053s __ 11m44.222s
>3 nodes -j 18 _________ 25m4.973s ____ 63m48.105s ___ 7m53.588s
>3 nodes -j 33 _________ 17m27.299s ___ 55m18.695s ___ 7m38.526s
>3 nodes -j 33, 25G net _ 17m0.235s _____ 46m24.425s ___ 7m13.615s
>.... + ram disk ________ 17m2.661s _____ 47m0.784s ____ 7m11.908s
.... + fix in boost ______ 13m22.895s ____ 45m16.003s ___ 4m55.285s -
almost 6 years ago by rpenyaev | Reply
Roman, can the following kernel patch also speedup our ceph builds? Seems worth to try
https://www.phoronix.com/scan.php?page=news_item&px=Linux-Pipe-Parallel-Job-Opt https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0ddad21d3e99c743a3aa473121dc5561679e26bb
-
Similar Projects
Improvements to osc (especially with regards to the Git workflow) by mcepl
Description
There is plenty of hacking on osc, where we could spent some fun time. I would like to see a solution for https://github.com/openSUSE/osc/issues/2006 (which is sufficiently non-serious, that it could be part of HackWeek project).
Try out Neovim Plugins supporting AI Providers by enavarro_suse
Description
Experiment with several Neovim plugins that integrate AI model providers such as Gemini and Ollama.
Goals
Evaluate how these plugins enhance the development workflow, how they differ in capabilities, and how smoothly they integrate into Neovim for day-to-day coding tasks.
Resources
- Neovim 0.11.5
- AI-enabled Neovim plugins:
- avante.nvim: https://github.com/yetone/avante.nvim
- Gp.nvim: https://github.com/Robitx/gp.nvim
- parrot.nvim: https://github.com/frankroeder/parrot.nvim
- gemini.nvim: https://dotfyle.com/plugins/kiddos/gemini.nvim
- ...
- Accounts or API keys for AI model providers.
- Local model serving setup (e.g., Ollama)
- Test projects or codebases for practical evaluation:
- OBS: https://build.opensuse.org/
- OBS blog and landing page: https://openbuildservice.org/
- ...
Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil
Join the Gitter channel! https://gitter.im/uyuni-project/hackweek
Uyuni is a configuration and infrastructure management tool that saves you time and headaches when you have to manage and update tens, hundreds or even thousands of machines. It also manages configuration, can run audits, build image containers, monitor and much more!
Currently there are a few distributions that are completely untested on Uyuni or SUSE Manager (AFAIK) or just not tested since a long time, and could be interesting knowing how hard would be working with them and, if possible, fix whatever is broken.
For newcomers, the easiest distributions are those based on DEB or RPM packages. Distributions with other package formats are doable, but will require adapting the Python and Java code to be able to sync and analyze such packages (and if salt does not support those packages, it will need changes as well). So if you want a distribution with other packages, make sure you are comfortable handling such changes.
No developer experience? No worries! We had non-developers contributors in the past, and we are ready to help as long as you are willing to learn. If you don't want to code at all, you can also help us preparing the documentation after someone else has the initial code ready, or you could also help with testing :-)
The idea is testing Salt (including bootstrapping with bootstrap script) and Salt-ssh clients
To consider that a distribution has basic support, we should cover at least (points 3-6 are to be tested for both salt minions and salt ssh minions):
- Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
- Onboarding (salt minion from UI, salt minion from bootstrap scritp, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
- Package management (install, remove, update...)
- Patching
- Applying any basic salt state (including a formula)
- Salt remote commands
- Bonus point: Java part for product identification, and monitoring enablement
- Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
- Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)
- Bonus point: testsuite enablement (https://github.com/uyuni-project/uyuni/tree/master/testsuite)
If something is breaking: we can try to fix it, but the main idea is research how supported it is right now. Beyond that it's up to each project member how much to hack :-)
- If you don't have knowledge about some of the steps: ask the team
- If you still don't know what to do: switch to another distribution and keep testing.
This card is for EVERYONE, not just developers. Seriously! We had people from other teams helping that were not developers, and added support for Debian and new SUSE Linux Enterprise and openSUSE Leap versions :-)
In progress/done for Hack Week 25
Guide
We started writin a Guide: Adding a new client GNU Linux distribution to Uyuni at https://github.com/uyuni-project/uyuni/wiki/Guide:-Adding-a-new-client-GNU-Linux-distribution-to-Uyuni, to make things easier for everyone, specially those not too familiar wht Uyuni or not technical.
openSUSE Leap 16.0
The distribution will all love!
https://en.opensuse.org/openSUSE:Roadmap#DRAFTScheduleforLeap16.0
Curent Status We started last year, it's complete now for Hack Week 25! :-D
[W]Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file) NOTE: Done, client tools for SLMicro6 are using as those for SLE16.0/openSUSE Leap 16.0 are not available yet[W]Onboarding (salt minion from UI, salt minion from bootstrap scritp, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)[W]Package management (install, remove, update...). Works, even reboot requirement detection
Switch software-o-o to store repomd in a database by hennevogel
Description
The openSUSE Software portal is a web app to explore binary packages of openSUSE distributions. Kind of like an package manager / app store.
https://software.opensuse.org/
This app has been around forever (August 2007) and it's architecture is a bit brittle. It acts as a frontend to the OBS distributions and published binary search APIs, calculates and caches a lot of stuff in memory and needs code changes nearly every openSUSE release to keep up.
As you can imagine, it's a heavy user of the OBS API, especially when caches are cold.
Goals
I want to change the app to cache repomod data in a (postgres) database structure
- Distributions have many Repositories
- Repositories have many Packages
- Packages have many Patches
The UI workflows will be as following
- As an admin I setup Distribution and it's repositories
- As an admin I sync all repositories repomd files into to the database
- As a user I browse a Distribution by category
- As a user I search for Package of a Distribution in it's Repositories
- As a user I extend the search to Package build on OBS for this Distribution
This has a couple of pro's:
- Less traffic on the OBS API as the usual Packages are inside the database
- Easier base to add features to this page. Like comments, ratings, openSUSE specific screenshots etc.
- Separating the Distribution package search from searching through OBS will hopefully make more clear for newbies that enabling extra repositories is kind of dangerous.
And one con:
- You can't search for packages build for foreign distributions with this app anymore (although we could consume their repomd etc. but I doubt we have the audience on an opensuse.org domain...)
TODO
Introduce a PG database- Add clockworkd as scheduler and delayed_job as ActiveJob backend
- Introduce ActiveStorage
- Build initial data model
- Introduce repomd to database sync
- Adapt repomd sync to Leap 16.0 repomod layout changes (single arch, no update repo)
Make repomd sync idempotent
- Introduce database search
- Setup foreman to run
rails sandrake jobs:workoff - Adapt UI
- Build Category Browsing
- Build Admin Distribution CRUD interface
Create a page with all devel:languages:perl packages and their versions by tinita
Description
Perl projects now live in git: https://src.opensuse.org/perl
It would be useful to have an easy way to check which version of which perl module is in devel:languages:perl. Also we have meta overrides and patches for various modules, and it would be good to have them at a central place, so it is easier to lookup, and we can share with other vendors.
I did some initial data dump here a while ago: https://github.com/perlpunk/cpan-meta
But I never had the time to automate this.
I can also use the data to check if there are necessary updates (currently it uses data from download.opensuse.org, so there is some delay and it depends on building).
Goals
- Have a script that updates a central repository (e.g.
https://src.opensuse.org/perl/_metadata) with metadata by looking at https://src.opensuse.org/perl/_ObsPrj (check if there are any changes from the last run) - Create a HTML page with the list of packages (use Javascript and some table library to make it easily searchable)
Resources
Results
Day 1
- First part of the code which retrieves data from https://src.opensuse.org/perl/_ObsPrj with submodules and creates a YAML and a JSON file.
- Repo: https://github.com/perlpunk/opensuse-perl-meta
- Also a first version of the HTML is live: https://perlpunk.github.io/opensuse-perl-meta/
Day 2
- HTML Page has now links to src.opensuse.org and the date of the last update, plus a short info at the top
- Code is now 100% covered by tests: https://app.codecov.io/gh/perlpunk/opensuse-perl-meta
- I used the modern perl
classfeature, which makes perl classes even nicer and shorter. See example - Tests
- I tried out the mocking feature of the modern Test2::V0 library which provides call tracking. See example
- I tried out comparing data structures with the new Test2::V0 library. It let's you compare parts of the structure with the
likefunction, which only compares the date that is mentioned in the expected data. example
Day 3
- Added various things to the table
- Dependencies column
- Show popup with info for cpanspec, patches and dependencies
- Added last date / commit to the data export.
Plan: With the added date / commit we can now daily check _ObsPrj for changes and only fetch the data for changed packages.
Day 4
Improvements to osc (especially with regards to the Git workflow) by mcepl
Description
There is plenty of hacking on osc, where we could spent some fun time. I would like to see a solution for https://github.com/openSUSE/osc/issues/2006 (which is sufficiently non-serious, that it could be part of HackWeek project).