Automate Haskell Packaging
an invention by psimons
Updated over 5 years ago. 3 hacker ♥️.

We have various individual tools to automate parts of the Haskell packaging process, like cabal-rpm, but those tools aren't integrated into a fully automated system that keeps Haskell packages up-to-date with as little human intervention as possible. I would like to build that system. Stackage provides us with an accurate list of packages and versions that are known to work together well, and there are basically two flavors: the nightly snapshot (bleeding edge) and the LTS release (stable API). The former is appropriate for Tumbleweed, IMHO, and the latter is appropriate for stable releases like SLE or Leap.

Now, we can use cabal-rpm to generate spec files automatically for all packages in a Stackage release and check them into OBS. The process does need some tweaking, however, because cabal-rpm generates spec files that don't always work well for SUSE. We could (a) branch cabal-rpm and add SUSE-specific know-how to remedy that issue or we could (b) maintain a set of patches that adapt the generated spec files to our needs. Once the Stackage releases are available in an OBS development project, we need another automated process that submits all updated packages to openSUSE:Factory, etc.

Getting that entire system up and running within a week is probably a bit ambitious, but it should be quite possible to achieve a working prototype that solves 70-90% of the tasks we do manually these days.

Looking for hackers with the skills:

haskell ci obs bash

This project is part of:

Hack Week 14

Activity

  • over 8 years ago: evshmarnev liked this project.
  • over 8 years ago: pluskalm liked this project.
  • over 8 years ago: osukup liked this project.
  • over 8 years ago: osukup joined this project.
  • over 8 years ago: psimons started this project.
  • over 8 years ago: psimons added keyword "haskell" to this project.
  • over 8 years ago: psimons added keyword "ci" to this project.
  • over 8 years ago: psimons added keyword "obs" to this project.
  • over 8 years ago: psimons added keyword "bash" to this project.
  • over 8 years ago: psimons originated this project.


    • Comments

    • osukup
      over 8 years ago by osukup | Reply

      @psimons prepared new way to automate creating haskell packages for rpm based distributions

      results are in opensuse-haskell github repository

      we moved to statically linked haskell applications

      and began with preparing OBS repositories for automated submitting haskell packages

      now wee need prepare config and patch files for differences between generated spec files and suse/distribution specific changes

    • psimons
      over 8 years ago by psimons | Reply

      The automated generation of OBS projects from a Hackage/Stackage config file works nicely. We have added a few thousand Haskell packages to OBS in https://build.opensuse.org/project/show/devel:languages:haskell:lts:6 (LTS Haskell 6.x) and https://build.opensuse.org/project/show/devel:languages:haskell:lts:next (Stackage Nightly). There is more effort necessary to clean up the build process and to create manual patches to remedy issues in the auto-generated spec files, but all things considered we were are quite happy with the results accomplished during Hackweek.

    Similar Projects

    Update Haskell ecosystem in Tumbleweed to GHC-9.10.x by psimons

    Description

    We are currently at GHC-9.8.x, which a bit old. So I'd like to take a shot at the latest version of the compiler, GHC-9.10.x. This is gonna be interesting because the new version requires major updates to all kinds of libraries and base packages, which typically means patching lots of packages to make them build again.

    Goals

    Have working builds of GHC-9.10.x and the required Haskell packages in 'devel:languages:haskell` so that we can compile:

    • git-annex
    • pandoc
    • xmonad
    • cabal-install

    Resources

    • https://build.opensuse.org/project/show/devel:languages:haskell/
    • https://github.com/opensuse-haskell/configuration/
    • #discuss-haskell
    • https://www.twitch.tv/peti343

    Drag Race - comparative performance testing for pull requests by balanza

    Description

    «Sophia, a backend developer, submitted a pull request with optimizations for a critical database query. Once she pushed her code, an automated load test ran, comparing her query against the main branch. Moments later, she saw a new comment automatically added to her PR: the comparison results showed reduced execution time and improved efficiency. Smiling, Sophia messaged her team, “Performance gains confirmed!”»

    Goals

    • To have a convenient and ergonomic framework to describe test scenarios, including environment and seed;
    • to compare results from different tests
    • to have a GitHub action that executes such tests on a CI environment

    Resources

    The MVP will be built on top of Preevy and K6.

    Implement a full OBS api client in Rust by nbelouin

    Description

    I recently started to work on tooling for OBS using rust, to do so I started a Rust create to interact with OBS API, I only implemented a few routes/resources for what I needed. What about making it a full fledged OBS client library.

    Goals

    • Implement more routes/resources
    • Implement a test suite against the actual OBS implementation
    • Bonus: Create an osc like cli in Rust using the library

    Resources

    • https://github.com/suse-edge/obs-tools/tree/main/obs-client
    • https://api.opensuse.org/apidocs/

    New features in openqa-trigger-from-obs for openQA by jlausuch

    Description

    Implement new features in openqa-trigger-from-obs to make xml more flexible.

    Goals

    One of the features to be implemented: - Possibility to define "VERSION" and "ARCH" variables per flavor instead of global.

    Resources

    https://github.com/os-autoinst/openqa-trigger-from-obs

    obs-service-vendor_node_modules by cdimonaco

    Description

    When building a javascript package for obs, one option is to use https://github.com/openSUSE/obs-service-node_modules as source service to get the project npm dependencies available for package bulding.

    obs-service-vendornodemodules aims to be a source service that vendors npm dependencies, installing them with npm install (optionally only production ones) and then creating a tar package of the installed dependencies.

    The tar will be used as source in the package building definitions.

    Goals

    • Create an obs service package that vendors the npm dependencies as tar archive.
    • Maybe add some macros to unpack the vendor package in the specfiles

    Resources

    Learn obs/ibs sync tool by xlai

    Description

    Once images/repo are built from IBS/OBS, there is a tool to sync the image from IBS/OBS to openqa asset directory and trigger openqa jobs accordingly.

    Goals

    Check how the tool is implemented, and be capable to add/modify our needed images/repo in future by ourselves.

    Resources

    • https://github.com/os-autoinst/openqa-trigger-from-obs
    • https://gitlab.suse.de/openqa/openqa-trigger-from-ibs-plugin/-/tree/master?ref_type=heads

    Learn about OSB and contribute to `kustomize` and `k9s` packages to add ARM arch by dpock

    Description

    There are already k9s and kustomize packages that exist for openSUSE today. These could be used as the source for these binaries in our rancher projects. By using them we would benefit from CVE fixes included in our distribution of the packages not in cluded upstream. However they are not providing arm package builds which are required.

    Goals

    • [ ] Update the kustomize package in OBS to use the newest version and send change request

    Resources

    • k9s: https://build.opensuse.org/package/show/openSUSE:Factory/k9s
    • kustomize: https://build.opensuse.org/package/show/openSUSE:Factory/kustomize
    • Learning Docs: https://confluence.suse.com/display/packaging/Training%2C+Talks+and+Videos

    Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil

    Join the Gitter channel! https://gitter.im/uyuni-project/hackweek

    Uyuni is a configuration and infrastructure management tool that saves you time and headaches when you have to manage and update tens, hundreds or even thousands of machines. It also manages configuration, can run audits, build image containers, monitor and much more!

    Currently there are a few distributions that are completely untested on Uyuni or SUSE Manager (AFAIK) or just not tested since a long time, and could be interesting knowing how hard would be working with them and, if possible, fix whatever is broken.

    For newcomers, the easiest distributions are those based on DEB or RPM packages. Distributions with other package formats are doable, but will require adapting the Python and Java code to be able to sync and analyze such packages (and if salt does not support those packages, it will need changes as well). So if you want a distribution with other packages, make sure you are comfortable handling such changes.

    No developer experience? No worries! We had non-developers contributors in the past, and we are ready to help as long as you are willing to learn. If you don't want to code at all, you can also help us preparing the documentation after someone else has the initial code ready, or you could also help with testing :-)

    The idea is testing Salt and Salt-ssh clients, but NOT traditional clients, which are deprecated.

    To consider that a distribution has basic support, we should cover at least (points 3-6 are to be tested for both salt minions and salt ssh minions):

    1. Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
    2. Onboarding (salt minion from UI, salt minion from bootstrap scritp, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
    3. Package management (install, remove, update...)
    4. Patching
    5. Applying any basic salt state (including a formula)
    6. Salt remote commands
    7. Bonus point: Java part for product identification, and monitoring enablement
    8. Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
    9. Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)
    10. Bonus point: testsuite enablement (https://github.com/uyuni-project/uyuni/tree/master/testsuite)

    If something is breaking: we can try to fix it, but the main idea is research how supported it is right now. Beyond that it's up to each project member how much to hack :-)

    • If you don't have knowledge about some of the steps: ask the team
    • If you still don't know what to do: switch to another distribution and keep testing.

    This card is for EVERYONE, not just developers. Seriously! We had people from other teams helping that were not developers, and added support for Debian and new SUSE Linux Enterprise and openSUSE Leap versions :-)

    Pending

    FUSS

    FUSS is a complete GNU/Linux solution (server, client and desktop/standalone) based on Debian for managing an educational network.

    https://fuss.bz.it/

    Seems to be a Debian 12 derivative, so adding it could be quite easy.

    • [W] Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
    • [W] Onboarding (salt minion from UI, salt minion from bootstrap script, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator) --> Working for all 3 options (salt minion UI, salt minion bootstrap script and salt-ssh minion from the UI).
    • [W] Package management (install, remove, update...) --> Installing a new package works, needs to test the rest.
    • [I] Patching (if patch information is available, could require writing some code to parse it, but IIRC we have support for Ubuntu already). No patches detected. Do we support patches for Debian at all?
    • [W] Applying any basic salt state (including a formula)
    • [W] Salt remote commands
    • [ ] Bonus point: Java part for product identification, and monitoring enablement