OpenQA test list views improvement
a project by mdati
Updated over 2 years ago. 2 hacker ♥️. 4 followers.

Project Description

Create views of OpenQA Test results in Grafana, grouped i.e. for Version, Builds, Flavor, Arch, in order to have a global view list, but also capability to explore details of each test, for a more integrated faults investigation environment.
Test results can be from OSD or OOO or any host.
Scheduled openqa tests details could be collected using openqa-cli api in JSON format.

Goal for this Hackweek

  • Study Grafana and eventual needed plugins
  • Identify a proper configuration/data-source/dashboard for this target
  • Prepare an initial test environment settings of openQA tests in Grafana
  • Possibly, display a basic test list in Grafana and
  • Possibly, add capability to inspect each test settings and results details

Resources

Grafana examples;
Grafana dashboards;
openqa REST API

keywords: grafana, openqa,

Join this project

Looking for hackers with the skills:

grafana openqa testing dashboard

This project is part of:

Hack Week 22

Activity

  • about 1 year ago: kennaanna joined this project.
  • over 2 years ago: mdati started this project.
  • over 2 years ago: ilausuch liked this project.
  • over 2 years ago: okurz liked this project.
  • over 2 years ago: mdati added keyword "grafana" to this project.
  • over 2 years ago: mdati added keyword "openqa" to this project.
  • over 2 years ago: mdati added keyword "testing" to this project.
  • over 2 years ago: mdati added keyword "dashboard" to this project.
  • over 2 years ago: mdati originated this project.


    • Comments

    Be the first to comment!

    Similar Projects

    Flaky Tests AI Finder for Uyuni and MLM Test Suites by oscar-barrios

    Description

    Our current Grafana dashboards provide a great overview of test suite health, including a panel for "Top failed tests." However, identifying which of these failures are due to legitimate bugs versus intermittent "flaky tests" is a manual, time-consuming process. These flaky tests erode trust in our test suites and slow down development.

    This project aims to build a simple but powerful Python script that automates flaky test detection. The script will directly query our Prometheus instance for the historical data of each failed test, using the jenkins_build_test_case_failure_age metric. It will then format this data and send it to the Gemini API with a carefully crafted prompt, asking it to identify which tests show a flaky pattern.

    The final output will be a clean JSON list of the most probable flaky tests, which can then be used to populate a new "Top Flaky Tests" panel in our existing Grafana test suite dashboard.

    Goals

    By the end of Hack Week, we aim to have a single, working Python script that:

    1. Connects to Prometheus and executes a query to fetch detailed test failure history.
    2. Processes the raw data into a format suitable for the Gemini API.
    3. Successfully calls the Gemini API with the data and a clear prompt.
    4. Parses the AI's response to extract a simple list of flaky tests.
    5. Saves the list to a JSON file that can be displayed in Grafana.
    6. New panel in our Dashboard listing the Flaky tests

    Resources