an invention by oscar-barrios
Project Description
Flaky tests: Those that have not a consistent result, and sometimes are failing and others are passing.
When we review a Test Report in a Pull Request, we would like to identify which of the test features or scenarios we are reviewing has been failing intermittently in the past, which will facilitate a comparison of results between our PR and the main branch.
To address that we want to:
Have a list of flaky test cases
- We can use the cards in the view https://github.com/orgs/SUSE/projects/23/views/7 (We can do API calls with graphql for that)
- We can try to use some algorithm based on the results stored in Prometheus
Inject Cucumber Tags into the .feature file (feature or scenario levels)
- In Runtime (not committing the change)
- That will allow us to show it in the HTML Test report later in a transparent way
Goal for this Hackweek
Show, in the HTML Test Report, a new Cucumber Tag "flaky_test" in those scenarios that the team identified as Flaky Tests.
Looking for hackers with the skills:
This project is part of:
Hack Week 23
Activity
Comments
-
10 months ago by oscar-barrios | Reply
The main code can be found in this PR: https://github.com/uyuni-project/uyuni/pull/7832 Additionally, there is a small change in our CI pipelines, in order to call a new Rake task defined in the PR above.
Similar Projects
This project is one of its kind!