Project Description

Flaky tests: Those that have not a consistent result, and sometimes are failing and others are passing.

When we review a Test Report in a Pull Request, we would like to identify which of the test features or scenarios we are reviewing has been failing intermittently in the past, which will facilitate a comparison of results between our PR and the main branch.

To address that we want to:

  • Have a list of flaky test cases

  • Inject Cucumber Tags into the .feature file (feature or scenario levels)

    • In Runtime (not committing the change)
    • That will allow us to show it in the HTML Test report later in a transparent way

Goal for this Hackweek

Show, in the HTML Test Report, a new Cucumber Tag "flaky_test" in those scenarios that the team identified as Flaky Tests.

Looking for hackers with the skills:

github graphql cucumber susemanager

This project is part of:

Hack Week 23

Activity

  • 6 months ago: jordimassaguerpla liked this project.
  • 7 months ago: oscar-barrios started this project.
  • 7 months ago: oscar-barrios added keyword "susemanager" to this project.
  • 7 months ago: oscar-barrios added keyword "github" to this project.
  • 7 months ago: oscar-barrios added keyword "graphql" to this project.
  • 7 months ago: oscar-barrios added keyword "cucumber" to this project.
  • 7 months ago: oscar-barrios originated this project.

  • Comments

    • oscar-barrios
      7 months ago by oscar-barrios | Reply

      The main code can be found in this PR: https://github.com/uyuni-project/uyuni/pull/7832 Additionally, there is a small change in our CI pipelines, in order to call a new Rake task defined in the PR above.