yast2-storage-ng as a libstorage-ng wrapper. POC
a project by ancorgs
Updated over 5 years ago. 1 hackers ♥️.

The goal of this project is to write a proof of concept of a new philosophy for yast2-storage-ng. Instead of just extending the API offered by libstorage-ng, the idea is wrap libstorage-ng so the Ruby code using yast2-storage-ng does not have direct visibility (unless explicitly desired) on the libstorage-ng classes and methods.

If you don't know what all that means, keep reading.

The YaST team is rewriting the storage stack (the part that manages disks, partitions, filesystems and so on). In the current implementation the low-level operations (partitioning and so on) are performed by a C++ library called libstorage. On top of that, yast2-storage provides an API for all the other YaST components and modules.

In the new implementation that should debut in SLE13, libstorage-ng already offers a higher level object oriented API with Ruby bindings (generated using SWIG). Although yast2-storage-ng is still there offering convenience classes and methods, now other YaST components directly use the classes provided by libstorage-ng to inspect the system and so on. So, contrary to the current implementation, yast2-storage-ng does not wrap libstorage-ng, but complements it by offering additional classes and refinements on top of the classes provided by libstorage-ng.

Now that libstorage-ng and yast2-storage-ng are quite feature-complete (although still far from the current versions) and used in several other YaST components, the direct use of libstorage-ng is starting to show some annoyances.

  • Sometimes, the strict type-checking imposed by SWIG forces the Ruby code using the library to look clumsy (explicit type casting, a lot of calls to #to_a in almost every collection coming from libstorage-ng, etc.).
  • The structure of libstorage-ng is very flexible and, by design, it never hides that to the user of the API. Offering a method like a_filesystem.disk would go against the philosophy of the library. The user of the library is expected to do something like a_filesystem.blk_devices[0].partition_table.partionable. That adds clarity about the real libstorage-ng structure, but does not ease the common case.
  • The C++ take on object orientation sometimes differs from the Ruby one. We have discussed for very long time and in many threads about several aspects of the libstorage-ng API without reaching a clear consensus between the two "sides" of the team - the developers with a solid background in C++ vs the Ruby purists.

Fixing those problems by adding many refinements feels wrong (refinements are still second-class citizens in the Ruby world). On the other hand, YaST does not need access to every single class provided by libstorage-ng, at least not in a regular basis.

So let's see go back to the wrapping approach and see what happens. I already tried in the past and SWIG was too much in the way. But now I have new ideas.

Join this project

Looking for hackers with the skills:

yast ruby c++

This project is part of:

Hack Week 15

Activity

  • almost 8 years ago: dwaas liked this project.
  • almost 8 years ago: ancorgs added keyword "yast" to this project.
  • almost 8 years ago: ancorgs added keyword "ruby" to this project.
  • almost 8 years ago: ancorgs added keyword "c++" to this project.
  • almost 8 years ago: ancorgs started this project.
  • almost 8 years ago: ancorgs originated this project.


    • Comments

    • ancorgs
      almost 8 years ago by ancorgs | Reply

      This pull request shows some working code and explains how the new layer works (or would work when finished). Used as base for discussing the concept with the whole YaST team. Despite the length of the thread in the yast-devel mailing list, the proposal was well accepted. 7 out of the 8 YaST developers who expressed their opinion (in the mailing list or the PR) like the new layer and think it's the way to go.

      I spent the last day of Hack Week analyzing how the current query interface is used from other YaST modules and how to adapt that query interface to the new layer. The conclusion is that we just need to add some methods to the new layer and then we can even drop the current query interface.

      So we just need to allocate some time to finish this.

    Similar Projects

    Recipes catalog and calculator in Rails 8 by gfilippetti

    My wife needs a website to catalog and sell the products of her upcoming bakery, and I need to learn and practice modern Rails. So I'm using this Hack Week to build a modern store using the latest Ruby on Rails best practices, ideally up to the deployment.

    TO DO

    • Index page
    • Product page
    • Admin area -- Supplies calculator based on orders -- Orders notification
    • Authentication
    • Payment
    • Deployment

    Day 1

    As my Rails knowledge was pretty outdated and I had 0 experience with Turbo (wich I want to use in the app), I started following a turbo-rails course. I completed 5 of 11 chapters.

    Day 2

    Continued the course until chapter 8 and added live updates & an empty state to the app. I should finish the course on day 3 and start my own project with the knowledge from it.

    Hackweek 24

    For this Hackweek I'll continue this project, focusing on a Catalog/Calculator for my wife's recipes so she can use for her Café.

    Day 1

    Fix RSpec tests in order to replace the ruby-ldap rubygem in OBS by enavarro_suse

    Description

    "LDAP mode is not official supported by OBS!". See: config/options.yml.example#L100-L102

    However, there is an RSpec file which tests LDAP mode in OBS. These tests use the ruby-ldap rubygem, mocking the results returned by a LDAP server.

    The ruby-ldap rubygem seems no longer maintaned, and also prevents from updating to a more recent Ruby version. A good alternative is to replace it with the net-ldap rubygem.

    Before replacing the ruby-ldap rubygem, we should modify the tests so the don't mock the responses of a LDAP server. Instead, we should modify the tests and run them against a real LDAP server.

    Goals

    Goals of this project:

    • Modify the RSpec tests and run them against a real LDAP server
    • Replace the net-ldap rubygem with the ruby-ldap rubygem

    Achieving the above mentioned goals will:

    • Permit upgrading OBS from Ruby 3.1 to Ruby 3.2
    • Make a step towards officially supporting LDAP in OBS.

    Resources

    YQPkg - Bringing the Single Package Selection Back to Life by shundhammer

    tl;dr

    Rip out the high-level YQPackageSelector widget from YaST and make it a standalone Qt program without any YaST dependencies.

    See section "Result" at the bottom for the current status after the hack week.

    Current Status

    See the development status issue at the GitHub repo.

    tl;dr: It's usable now with all the key features.

    It does real package installation / removal / update with reasonable user feedback.

    The Past and the Present

    We used to have and still have a powerful software selection with the YaST sw_single module (and the YaST patterns counterpart): You can select software down to the package level, you can easily select one of many available package versions, you can select entire patterns - or just view them and pick individual packages from patterns.

    You can search packages based on name, description, "requires" or "provides" level, and many more things.

    The Future

    YaST is on its way out, to be replaced by the new Agama installer and Cockpit for system administration. Those tools can do many things, but fine-grained package selection is not among them. And there are also no other Open Source tools available for that purpose that even come close to the YaST package selection.

    Many aspects of YaST have become obsolete over the years; many subsystems now come with a good default configuration, or they can configure themselves automatically. Just think about sound or X11 configuration; when did you last need to touch them?

    For others, the desktops bring their own tools (e.g. printers), or there are FOSS configuration tools (NetworkManager, BlueMan). Most YaST modules are no longer needed, and for many others there is a replacement in tools like Cockpit.

    But no longer having a powerful fine-grained package selection like in YaST sw_single will hurt. Big time. At least until there is an adequate replacement, many users will want to keep it.

    The Idea

    YaST sw_single always revolved around a powerful high-level widget on the abstract UI level. Libyui has low-level widgets like YPushButton, YCheckBox, YInputField, more advanced ones like YTable, YTree; and some few very high-level ones like YPackageSelector and YPatternSelector that do the whole package selection thing alone, working just on the libzypp level and changing the status of packages or patterns there.

    For the YaST Qt UI, the YQPackageSelector / YQPatternSelector widgets work purely on the Qt and libzypp level; no other YaST infrastructure involved, in particular no Ruby (or formerly YCP) interpreter, no libyui-level widgets, no bindings between Qt / C++ and Ruby / YaST-core, nothing. So it's not too hard to rip all that part out of YaST and create a standalone program from it.

    For the NCurses UI, the NCPackageSelector / NCPatternSelector create a lot of libyui widgets (inheriting YWidget / NCWidget) and use a lot of libyui calls to glue them together; and all that of course still needs a lot of YaST / libyui / libyui-ncurses infrastructure. So NCurses is out of scope here.

    Preparatory Work: Initializing the Package Subsystem

    To see if this is feasible at all, the existing UI examples needed some fixing to check what is needed on that level. That was the make-or-break decision: Would it be realistically possible to set the needed environment in libzypp up (without being stranded in the middle of that task alone at the end of the hack week)?

    Yes, it is: That part is already working:

    https://github.com/yast/yast-ycp-ui-bindings/pull/71

    Port some classic game to Linux by MDoucha

    Let's pick some old classic game, reverse engineer the data formats and game rules and write an open source engine for it from scratch. Some games from 1990s are simple enough that we could have a playable prototype by the end of the week.

    Write which games you'd like to hack on in the comments. Don't forget to check e.g. on Open Source Game Clones, Github and SourceForge whether the game is ported already.

    Hack Week 24 - Master of Orion II: Battle at Antares & Chaos Overlords

    Work on Master of Orion II continues but we can hack more than one game. Chaos Overlords is a dystopian, lighthearted, cyberpunk turn-based strategy game originally released in 1996 for Windows 95 and Mac OS. The player takes on the role of a Chaos Overlord, attempting to control a city. Gameplay involves hiring mercenary gangs and deploying them on an 8-by-8 grid of city sectors to generate income, occupy sectors and take over the city.

    How to ~~install & play~~ observe the decompilation progress:

    • Clone the Git repository
    • A playable reimplementation does not exist yet, but when it does, it will be linked in the repository mentioned above.

    Further work needed:

    • Analyze the remaining unknown data structures, most of which are related to the AI.
    • Decompile the AI completely. The strong AI is part of the appeal of the game. It cannot be left out.
    • Reimplement the game.

    Hack Week 20, 21, 22 & 23 - Master of Orion II: Battle at Antares

    Master of Orion II is one of the greatest turn-based 4X games of the 1990s. Explore the galaxy, colonize planets, research new technologies, fight space monsters and alien empires and in the end, become the ruler of the galaxy one way or another.

    How to install & play:

    • Clone the Git repository
    • Run ./bootstrap; ./configure; make && make install
    • Copy all *.LBX files from the original Master of Orion II to the installation data directory (/usr/local/share/openorion2 by default)
    • Run openorion2

    Further work needed:

    • Analyze the rest of the original savegame format and a few remaining data files.
    • Implement most of the game. The open source engine currently supports only loading saved games from the original version and viewing the galaxy map, fleet management and list of known planets.

    Hack Week 19 - Signus: The Artifact Wars

    Signus is a Czech turn-based strategy game similar to Panzer General or Battle Isle series. Originally published in 1998 and open-sourced by the original developers in 2003.

    How to install & play:

    • Clone the Git repository
    • Run ./bootstrap; ./configure; make && make install in both signus and signus-data directories.
    • Run signus

    Further work needed:

    • Create openSUSE package
    • Implement full support for original game data (the open source version uses slightly different data file contents but original game data can be converted using a script).

    New KDE Plasma notification app/applet by apappas

    Description

    My memory is terrible so I depend a lot on notifications to carry me through the workday. As a plasma user I am ok with the current applet, but I don't love it. It is too small for the centrality it has in my day. Also I dislike how you can not go back to notifications you have dismissed

    Goals

    Develop a plasma app that * must gather notifications without disrupting the existing notification app * must offer the ablity to refer to dismissed/archived/seen notification up to some defined point in the past * must allow deletion of notifications

    RISC-V emulator in GLSL capable of running Linux by favogt

    Description

    There are already numerous ways to run Linux and some programs through emulation in a web browser (e.g. x86 and riscv64 on https://bellard.org/jslinux/), but none use WebGL/WebGPU to run the emulation on the GPU.

    I already made a PoC of an AArch64 (64-bit Arm) emulator in OpenCL which is unfortunately hindered by a multitude of OpenCL compiler bugs on all platforms (Intel with beignet or the new compute runtime and AMD with Mesa Clover and rusticl). With more widespread and thus less broken GLSL vs. OpenCL and the less complex implementation requirements for RV32 (especially 32bit integers instead of 64bit), that should not be a major problem anymore.

    Goals

    Write an RISC-V system emulator in GLSL that is capable of booting Linux and run some userspace programs interactively. Ideally it is small enough to work on online test platforms like Shaderoo with a custom texture that contains bootstrap code, kernel and initrd.

    Minimum:

    riscv32 without FPU (RV32 IMA) and MMU (µClinux), running Linux in M-mode and userspace in U-mode.

    Stretch goals:

    FPU support, S-Mode support with MMU, SMP. Custom web frontend with more possibilities for I/O (disk image, network?).

    Resources

    RISC-V ISA Specifications
    Shaderoo
    OpenGL 4.5 Quick Reference Card

    Result as of Hackweek 2024

    WebGL turned out to be insufficient, it only supports OpenGL ES 3.0 but imageLoad/imageStore needs ES 3.1. So we switched directions and had to write a native C++ host for the shaders.

    As of Hackweek Friday, the kernel attempts to boot and outputs messages, but panics due to missing memory regions.

    Since then, some bugs were fixed and enough hardware emulation implemented, so that now Linux boots with framebuffer support and it's possible to log in and run programs!

    The repo with a demo video is available at https://github.com/Vogtinator/risky-v