an invention by oscar-barrios
Project Description
Flaky tests: Those that have not a consistent result, and sometimes are failing and others are passing.
When we review a Test Report in a Pull Request, we would like to identify which of the test features or scenarios we are reviewing has been failing intermittently in the past, which will facilitate a comparison of results between our PR and the main branch.
To address that we want to:
Have a list of flaky test cases
- We can use the cards in the view https://github.com/orgs/SUSE/projects/23/views/7 (We can do API calls with graphql for that)
- We can try to use some algorithm based on the results stored in Prometheus
Inject Cucumber Tags into the .feature file (feature or scenario levels)
- In Runtime (not committing the change)
- That will allow us to show it in the HTML Test report later in a transparent way
Goal for this Hackweek
Show, in the HTML Test Report, a new Cucumber Tag "flaky_test" in those scenarios that the team identified as Flaky Tests.
Looking for hackers with the skills:
This project is part of:
Hack Week 23
Activity
Comments
-
about 1 month ago by oscar-barrios | Reply
The main code can be found in this PR: https://github.com/uyuni-project/uyuni/pull/7832 Additionally, there is a small change in our CI pipelines, in order to call a new Rake task defined in the PR above.
Similar Projects
Test Results for openQA on GitHub by livdywan
Project Description
Jobs in openQA are us...
Avahi Integration and Network Connection by vojha
Avahi Integration and Network Connection
...
Deep clean-up of the Uyuni documentation files by omaric
Project Description
This project is plann...
Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil
Join the Gitter channel! [https://gitter.im/uy...
Saline (state deployment control and monitoring tool for SUSE Manager/Uyuni) by vizhestkov
[comment]: # (Please use the project descriptio...
Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil
Join the Gitter channel! [https://gitter.im/uy...
Enhance product migration feature in Uyuni(SUSE Manager) to migrate from regular SLES to SLES4SAP by admehmood
[comment]: # (Please use the project descriptio...
Integration of Ironic in Uyuni for automating provisioning of bare metal machines. by ygutierrez
[comment]: # (Please use the project descriptio...
Create tool to analyze supportconfig to spot common SUSE Manager / Uyuni issues by cbosdonnat
[comment]: # (Please use the project descriptio...