Project Description

Most developers are comfortable with the workflows of git hosting services like gitlab and github, including their CI/CD capabilities. This project aims to experiment with new downstream package development and maintenance workflows based on upstream git repositories cloned at gitlab.suse.de. I'll be using the libvirt package for these experiments since it typically contains a healthy mixture of downstream-only patches along with upstream cherry picks.

I'd like to hide as much obs/ibs as I can behind some helper scripts for use by the package maintainer. Package developers and contributors would use the standard git workflows and could conveniently ignore the build service. The kernel package is developed in a similar fashion. I'd like to create a simple template that is usable by less complex packages.

Goals for this Hackweek

Create some helpers that a package maintainer can use to generate the artifacts needed for an obs/ibs package submission from an upstream git repository.

Create gitlab CI configuration and scripts that can be used to execute tests on remote runners in the Provo devlab.

Resources

https://gitlab.suse.de/virtualization/libvirt

https://gitlab.suse.de/virtualization/libvirt-rpm

Looking for hackers with the skills:

obs gitlab-ci git gitlab buildservice

This project is part of:

Hack Week 21

Activity

  • over 1 year ago: archanaserver liked this project.
  • over 1 year ago: archanaserver joined this project.
  • over 1 year ago: jzerebecki liked this project.
  • almost 2 years ago: jfehlig added keyword "buildservice" to this project.
  • almost 2 years ago: jfehlig added keyword "git" to this project.
  • almost 2 years ago: jfehlig added keyword "gitlab" to this project.
  • almost 2 years ago: jfehlig removed keyword ci/cd from this project.
  • almost 2 years ago: jfehlig added keyword "gitlab-ci" to this project.
  • almost 2 years ago: jfehlig added keyword "ci/cd" to this project.
  • almost 2 years ago: jfehlig added keyword "obs" to this project.
  • almost 2 years ago: jfehlig started this project.
  • almost 2 years ago: jfehlig originated this project.

  • Comments

    • jfehlig
      over 1 year ago by jfehlig | Reply

      Illness and life-belated project update:

      Upstream libvirt is mirrored from gitlab.com to gitlab.suse.de via a systemd timer service running in a VM in the Provo devlab. Other virtualization-related projects also mirror their upstream repositories in a similar fashion. For this reason the VM is called "code-mirror". Developers and contributors are free to work with these mirrored repositories using common git workflows.

      Products, including Factory/Tumbleweed, are maintained in branches based on upstream release tags. E.g. for SLE15 SP4, which contains libvirt 8.0.0, the gitlab.suse.de libvirt mirror contains a 'v8.0.0-sle15sp4' branch where maintenance can occur. The branch is based on the upstream v8.0.0 tag and contains SUSE-specific downstream patches and upstream cherry picks. Fixing a SLE15 SP4 bug generally means checking out the 'v8.0.0-sle15sp4' branch, cherry picking the necessary commit(s) from 'upstream-master', and pushing back to the 'v8.0.0-sle15sp4' on gitlab.suse.de.

      Downstream-only content such as build service and package management artifacts are maintained in a separate git repository on gitlab.suse.de named libvirt-rpm. This includes a downstream-specific CI/CD config file (aka .gitlab-ci.yml), a spec file template, the changelog, lintrc files, miscellaneous downstream source files, and any scripts needed to implement the build, deploy, test, and submit stages of a CI pipeline. The libvirt-rpm repository contains maintenance branches that share the same name as the maintenance branches in the main libvirt repository, allowing maintenance of the downstream-only content on a per-product basis as well.

      When creating a new maintenance branch in the main libvirt repository, the first commit in the branch is to add the libvirt-rpm repository as a submodule under the rpm directory. The second commit sets the submodule branch to the same name as the new maintenance branch. Subsequent commits include downstream patches and upstream cherry picks needed for the maintained product.

      When pushing a change to a maintenance branch, a new CI pipeline is triggered based on the instructions in the custom .gitlab-ci.yml. The first stage of the pipeline builds a package. The second stage deploys the package to a candidate host, the third stage tests the package, and finally the last stage submits the package to the associated obs/ibs devel project. Submitting the the package from the devel project to the target product is deferred to the package maintainer.

      The first stage of the pipeline is the most interesting, particularly extracting the artifacts needed for a package build from a branch in the mirrored upstream repository. On a push or merge request event, gitlab will clone the associated branch (and optionally recurse submodules) at $CIPROJECTDIR on a gitlab runner. For the libvirt project, this means the libvirt-rpm submodule is also expanded on the runner, making the spec file template, changelog, CI/CD scripts, and other downstream-only content available. One of the CI/CD scripts prepares the build service artifacts in a workspace by first creating an archive based on the libvirt release tag. This is essentially a pristine tarball of an upstream release. All downstream and cherry picked patches are then extracted from the branch. A spec file template is then populated with the list of extracted patches. Other build service artifacts are copied from the libvirt-rpm submodule to the workspace, where finally 'osc build' is invoked.

      Nothing precludes using the gitlab.suse.de libvirt repository to develop upstream features and bug fixes. Development branches can be pushed to the repository for safe keeping until they are suitable for upstream submission. Once committed upstream and mirrored back to the gitlab.suse.de repository, the commit(s) can be cherry picked to the product maintenance branches.

      https://gitlab.suse.de/virtualization/libvirt https://gitlab.suse.de/virtualization/libvirt-rpm

    Similar Projects

    Adapt Bootstrap code in OBS to support theming by enavarro_suse

    Project Description

    After the release of ...


    Improve database_cleaner.rb script in OBS by enavarro_suse

    Project Description

    There is some code to...


    Reduce the amount of TODOs for RuboCop in OBS by enavarro_suse

    Project Description

    The OBS project has a...


    Support for OVA build in OBS and better support for vmdk disks in kiwi by gmoro

    Project Description

    Implement support for O...


    Elixir LiveView clone of Etherpad (running on ALP) by socon

    Project Description

    Etherpad (etherpad.org)...