Many projects rely heavily on CI jobs, e.g. based on github actions. We already had ideas for tight integration of openQA into such workflows for years, e.g. in


  • G1: A pull request in github triggers actual openQA tests



Day 1-2

Unfortunately I could not invest any reasonable effort. for this project on the first two days of HackWeek so I could only quickly gather what we have done for related work in the past: In with we extended openqa-clone-custom-git-refspec to parse github pull request descriptions for special commands. We never came to use it in automatically triggered CI jobs and likely many have forgotten about this feature or never knew it existed in the first place

Day 3

Brainstormed with szarate about related work. Found and Put it into os-autoinst-distri-example github action workflow. Created an openqa-mon container build OBS project with:

osc meta prj home:okurz:container:openqa-mon -e
osc meta pkg home:okurz:container:openqa-mon openqa-mon -e
cd ~/local/osc/
osc co home:okurz:container:openqa-mon
vim Dockerfile
osc add Dockerfile
osc ci -m "Update Dockerfile"

After a quick waiting time we can now do

podman pull

or for a complete command

podman run --rm -it openqa-mon --continuous 10 --exit --follow

Found a typo in openqa-mon, fixed with

I will have yet to see if the command also works in a non-interactive github action environment.

Then I found so likely I don't need the other project anymore. I can use

podman run --rm -it openqa-mon --continuous 10 --exit --follow

I added three secrets into "OPENQAAPIUSER", "OPENQAAPIKEY", "OPENQAAPISECRET" which I read out from the o3 database. Following I can now trigger openQA jobs from github actions with curl -u $OPENQA_API_USER...

To be able to test in my fork without creating and updating pull requests I need those variables in my own scope as well so I added the variables to as well.

My development workflow looks like this: In vim I edit the workflow file, save and call !git amend -a && git rp using my personal git aliases, e.g. see, and then monitor which updates without reloading. I just need to open a browser tab for details on each run, e.g. shows a successful trigger of an openQA job and outputting the URL. is the result on o3. The github variables that should be used to construct a proper test name and build could apparently not be read. shows the triggering&monitoring of a job.

In I asked already about two years ago to have a nice way to use openqa-mon to monitor and output the aggregated result of jobs. It seems there is a problem though as openqa-mon --continuous 10 --exit --follow returns an exit code of zero for that incomplete openQA job when it should be non-zero according to Reported that in . I can consider using as alternative.

Regarding the default github variables seems I got it wrong. shows that I need to use $GITHUB_REPOSITORY, not ${{vars.GITHUB_REPOSITORY}} is showing good success now with a custom job state&result polling using:

while :; do read state result < <(echo $(curl -sS $job | jq -r .job.state,.job.result)) && [[ $state = 'done' ]] && break || echo "job state of job ID $job_i     d : $state, waiting…" && sleep 10; done

this waits correctly for the job to finish and return with the exit code corresponding to if the job passed or not:

job state of job ID 3088453 : scheduled, waiting…
job state of job ID 3088453 : running, waiting…
job ID 3088453 : state: done, result: incomplete
Error: Process completed with exit code 1.

Now the next step is to fix the openQA job so that it actually does something more useful, either for this repository or immediately in

Day 4

jobs like incomplete with the message

Reason: setup failure: The source directory /var/lib/openqa/cache/openqa1-opensuse/tests/example/needles does not exist

which also aborts the openQA job without even saving the vars.json which could have helped the user to debug.

Maybe with os-autoinst-distri-openQA I am more successful.

isotovideo -d CASEDIR= DISTRI=openqa NEEDLES_DIR= SCHEDULE=tests/install/boot

locally at least looks fine.

Small improvement PR

But back to the example distri.

isotovideo -d CASEDIR= DISTRI=example NEEDLES_DIR=needles SCHEDULE=tests/boot

looks fine as well. As expected starting the module "boot" and failing after 30s timeout as there are no needles.

So triggered again from github action as yesterday but with DISTRI=example NEEDLES_DIR=needles and then monitoring

but this still shows

Reason: setup failure: The source directory /var/lib/openqa/cache/openqa1-opensuse/tests/example/needles does not exist

To debug and test I am disabling normal worker classes on openqaworker20 in /etc/openqa/workers.ini and trying again with explicitly triggering on WORKER_CLASS=openqaworker20. I am still struggling with the needles dir as there is no copy of os-autoinst-distri-example on our o3 infra in the cache. and explains that I should be able to use NEEDLES_DIR=%CASEDIR%/needles. With curl %CASEDIR% turns into broken characters. Likely I need to escape the %CA… but did not succeed so far. But also

openqa-cli api --o3 -X POST jobs CASEDIR=https// TEST=test-push-okurz/os-autoinst-distri-example#feature/hackweek22_trigger_openqa_in_ci BUILD=okurz/os-autoinst-distri-example#feature/hackweek22_trigger_openqa_in_ci DISTRI=example WORKER_CLASS=openqaworker20 NEEDLES_DIR=%CASEDIR%/needles

is not helpful ending up in the same error and a weird replacement leading to URL + "/needles" referencing a local directory. Might work in isotovideo but I did not get that far.

It looks like we have the following problems here: 1. test distributions that have an in-repo needle link like os-autoinst-distri-example in combination with caching openQA workers and in combination with using a CASEDIR pointing to a git repo currently can not be triggered without error leading to "setup failure: The source directory … does not exist". NEEDLES_DIR=%CASEDIR%/needlesshould work here but the variable expansion happens before Worker/Engines/ looks for%CASEDIR%in the variable value which is not there anymore at that time 2. Even if the URL pointing to the same test distribution to be used for needles would come out fine os-autoinst can not parse url+branch+relative folder 3. curl messes up the%CASEDIR%` so I would need to find proper escaping

Maybe it's good if I continue to find a way to define a custom github action that I can reuse in other test distributions, e.g. openQA-in-openQA

Added according secrets in .

So find runs in now

Oh, and of course finally I find this one git branch where I already did more or less the same experiment but 2 years ago: facepalm. At least this includes the "trigger something on openQA from github actions" but of course there is more to it.

Day 5

Found to ease testing of github actions. Thinking to move the workflow definition to an github action repo similar as we did with I tried to install act as "gh extension" following but that failed with "failed to run extension: fork/exec /home/okurz/.local/share/gh/extensions/gh-act/gh-act: no such file or directory". I reported, maybe I am doing it wrong.

So I downloaded the binary into /usr/local/bin and execute from there. Now executing

while_inotifywait timeout 30 act -s OPENQA_TOKEN

but somehow this never comes to the point that the curl-call would return, kinda hangs there. Well, at least it's usable for a dry-run. Just to get openQA trigger variables right I can use the shell commands by themselves:

OPENQA_HOST= && job_id=$(curl -sS -u $OPENQA_TOKEN -X POST -d CASEDIR= -d TEST=test-push-os-autoinst/os-autoinst-distri-openQA#feature/hackweek22_trigger_openqa_in_ci -d BUILD=os-autoinst/os-autoinst-distri-openQA#feature/hackweek22_trigger_openqa_in_ci -d DISTRI=openqa -d NEEDLES_DIR=openqa/needles | jq .id) || echo "Failed: $job_id" && job=$OPENQA_HOST/api/v1/jobs/$job_id && while :; do read state result < <(echo $(curl -sS $job | jq -r .job.state,.job.result)) && [[ $state = 'done' ]] && break || echo "job state of job ID $job_id : $state, waiting…" && sleep 10; done && echo "job ID $job_id : state: $state, result: $result" && [[ $result = 'passed' ]]

this works to trigger and start a test which then later fails because we never specified an HDD. Cloning an existing job would work for this case but then the tricky part is to find the most recent job as template. For openQA we have but this is also not the most straight-forward approach to general. We could try to teach openqa-clone-job to work with the "latest" links.

I can use HDD_1_URL= but this is missing a gnome environment which so far in the openQA-in-openQA tests is required. We could work around that by setting up an openQA job, e.g. as part of the usual Tumbleweed snapshot tests, that triggers the upload of a fixed-named HDD image that we can use then.

By the way the above only works because there is a directory /var/lib/openqa/share/tests/openqa hence the DISTRI=openqa and NEEDLES_DIR=openqa/needles. That wouldn't work for a test distribution that is not already provided within that folder of openQA.


I managed to add GitHub action workflows that trigger and monitor openQA jobs on every push or pull request. Theoretically they would also trigger on new or edited comments in a pull request, a concept that could be explored further. I did not manage to come up with a generic way to trigger openQA jobs from an arbitrary repository due to limits in what openQA currently supports. So I can state that I reached the goal as originally defined although that was not reaching very far. I could identify limits and hence points where we could continue the work for openQA development as this HackWeek project was rather near to what we usually do within the SUSE QE Tools team anyway. Suggestions for features or bug fixes to look into:

  1. Support openQA test distributions with a simple needle subfolder included within the test distribution, e.g. os-autoinst-distri-example. At best those repositories should support simple jobs post calls with custom git repos like CASEDIR= and no need to specify variables just to "fix" the use of needles
  2. Learn how to escape %CASEDIR% to be able to use curl
  3. Extend openqa-clone-job to understand links like openqa-clone-job --within-instance ""
  4. In openQA support triggering test distributions with CASEDIR pointing to a git repo when the test distribution and according needles are not provided already on that openQA instance
  5. Generalize the action into a reusable action in its own repo, extend with instructions how to use

Looking for hackers with the skills:

Nothing? Add some keywords!

This project is part of:

Hack Week 22


  • over 1 year ago: jad12 liked this project.
  • over 1 year ago: okurz started this project.
  • over 1 year ago: mkoutny liked this project.
  • over 1 year ago: szarate liked this project.
  • over 1 year ago: jzerebecki liked this project.
  • over 1 year ago: hennevogel liked this project.
  • over 1 year ago: dancermak liked this project.
  • over 1 year ago: okurz originated this project.

  • Comments

    • hennevogel
      over 1 year ago by hennevogel | Reply

      If you could make this generic enough so we can do the same integration with OBS Submit Requests I will grateful :)

    Similar Projects

    This project is one of its kind!