an invention by oscar-barrios
Project Description
Flaky tests: Those that have not a consistent result, and sometimes are failing and others are passing.
When we review a Test Report in a Pull Request, we would like to identify which of the test features or scenarios we are reviewing has been failing intermittently in the past, which will facilitate a comparison of results between our PR and the main branch.
To address that we want to:
Have a list of flaky test cases
- We can use the cards in the view https://github.com/orgs/SUSE/projects/23/views/7 (We can do API calls with graphql for that)
- We can try to use some algorithm based on the results stored in Prometheus
Inject Cucumber Tags into the .feature file (feature or scenario levels)
- In Runtime (not committing the change)
- That will allow us to show it in the HTML Test report later in a transparent way
Goal for this Hackweek
Show, in the HTML Test Report, a new Cucumber Tag "flaky_test" in those scenarios that the team identified as Flaky Tests.
Looking for hackers with the skills:
This project is part of:
Hack Week 23
Activity
Comments
-
27 days ago by oscar-barrios | Reply
The main code can be found in this PR: https://github.com/uyuni-project/uyuni/pull/7832 Additionally, there is a small change in our CI pipelines, in order to call a new Rake task defined in the PR above.
Similar Projects
Test Results for openQA on GitHub by livdywan
Project Description
Jobs in openQA are us...
Deep clean-up of the Uyuni documentation files by omaric
Project Description
This project is plann...
Avahi Integration and Network Connection by vojha
Avahi Integration and Network Connection
...
Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil
Join the Gitter channel! [https://gitter.im/uy...
Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil
Join the Gitter channel! [https://gitter.im/uy...
Saline (state deployment control and monitoring tool for SUSE Manager/Uyuni) by vizhestkov
[comment]: # (Please use the project descriptio...
Integration of Ironic in Uyuni for automating provisioning of bare metal machines. by ygutierrez
[comment]: # (Please use the project descriptio...
Create tool to analyze supportconfig to spot common SUSE Manager / Uyuni issues by cbosdonnat
[comment]: # (Please use the project descriptio...
Enhance product migration feature in Uyuni(SUSE Manager) to migrate from regular SLES to SLES4SAP by admehmood
[comment]: # (Please use the project descriptio...