Flaky Tests AI Finder for Uyuni and MLM Test Suites by oscar-barrios

Description

Our current Grafana dashboards provide a great overview of test suite health, including a panel for "Top failed tests." However, identifying which of these failures are due to legitimate bugs versus intermittent "flaky tests" is a manual, time-consuming process. These flaky tests erode trust in our test suites and slow down development.

This project aims to build a simple but powerful Python script that automates flaky test detection. The script will directly query our Prometheus instance for the historical data of each failed test, using the jenkins_build_test_case_failure_age metric. It will then format this data and send it to the Gemini API with a carefully crafted prompt, asking it to identify which tests show a flaky pattern.

The final output will be a clean JSON list of the most probable flaky tests, which can then be used to populate a new "Top Flaky Tests" panel in our existing Grafana test suite dashboard.

Goals

By the end of Hack Week, we aim to have a single, working Python script that:

1. Connects to Prometheus and executes a query to fetch detailed test failure history.
2. Processes the raw data into a format suitable for the Gemini API.
3. Successfully calls the Gemini API with the data and a clear prompt.
4. Parses the AI's response to extract a simple list of flaky tests.
5. Saves the list to a JSON file that can be displayed in Grafana.
6. New panel in our Dashboard listing the Flaky tests

Resources

• Jenkins Prometheus Exporter: https://github.com/uyuni-project/jenkins-exporter/
• Data Source: Our internal Prometheus server.
• Key Metric: jenkins_build_test_case_failure_age{jobname, buildid, suite, case, status, failedsince}.
• Existing Query for Reference: count by (suite) (max_over_time(jenkins_build_test_case_failure_age{status=~"FAILED|REGRESSION", jobname="$jobname"}[$__range])).
• AI Model: The Google Gemini API.
• Example about how to interact with Gemini API: https://github.com/srbarrios/FailTale/
• Visualization: Our internal Grafana Dashboard.
• Internal IaC: https://gitlab.suse.de/galaxy/infrastructure/-/tree/master/srv/salt/monitoring

---

AI-Powered Unit Test Automation for Agama by joseivanlopez

The Agama project is a multi-language Linux installer that leverages the distinct strengths of several key technologies:

• Rust: Used for the back-end services and the core HTTP API, providing performance and safety.
• TypeScript (React/PatternFly): Powers the modern web user interface (UI), ensuring a consistent and responsive user experience.
• Ruby: Integrates existing, robust YaST libraries (e.g., yast-storage-ng) to reuse established functionality.

The Problem: Testing Overhead

Developing and maintaining code across these three languages requires a significant, tedious effort in writing, reviewing, and updating unit tests for each component. This high cost of testing is a drain on developer resources and can slow down the project's evolution.

The Solution: AI-Driven Automation

This project aims to eliminate the manual overhead of unit testing by exploring and integrating AI-driven code generation tools. We will investigate how AI can:

1. Automatically generate new unit tests as code is developed.
2. Intelligently correct and update existing unit tests when the application code changes.

By automating this crucial but monotonous task, we can free developers to focus on feature implementation and significantly improve the speed and maintainability of the Agama codebase.

Goals

• Proof of Concept: Successfully integrate and demonstrate an authorized AI tool (e.g., gemini-cli) to automatically generate unit tests.
• Workflow Integration: Define and document a new unit test automation workflow that seamlessly integrates the selected AI tool into the existing Agama development pipeline.
• Knowledge Sharing: Establish a set of best practices for using AI in code generation, sharing the learned expertise with the broader team.

Contribution & Resources

We are seeking contributors interested in AI-powered development and improving developer efficiency. Whether you have previous experience with code generation tools or are eager to learn, your participation is highly valuable.

If you want to dive deep into AI for software quality, please reach out and join the effort!

• Authorized AI Tools: Tools supported by SUSE (e.g., gemini-cli)
• Focus Areas: Rust, TypeScript, and Ruby components within the Agama project.

Interesting Links

• goose

---

Gemini-Powered Socratic Bug Evaluation and Management Assistant by rtsvetkov

Description

To build a tool or system that takes a raw bug report (including error messages and context) and uses a large language model (LLM) to generate a series of structured, Socratic-style questions designed to guide a the integration and development toward the root cause, rather than just providing a direct, potentially incorrect fix.

Goals

Set up a Python environment

Set the environment and get a Gemini API key. 2. Collect 5-10 realistic bug reports (from open-source projects, personal projects, or public forums like Stack Overflow—include the error message and the initial context).

Build the Dialogue Loop

1. Write a basic Python script using the Gemini API.
2. Implement a simple conversational loop: User Input (Bug) -> AI Output (Question) -> User Input (Answer to AI's question) -> AI Output (Next Question). Code Implementation

Socratic Strategy Implementation

1. Refine the logic to ensure the questions follow a Socratic path (e.g., from symptom-> context -> assumptions -> root cause).
2. Implement Function Calling (an advanced feature of the Gemini API) to suggest specific actions to the user, like "Run a ping test" or "Check the database logs."

Resources

---

Update M2Crypto by mcepl

There are couple of projects I work on, which need my attention and putting them to shape:

• M2Crypto

Goal for this Hackweek

• Put M2Crypto into better shape (most issues closed, all pull requests processed)
• More fun to learn jujutsu
• Play more with Gemini, how much it help (or not).
• Perhaps, also (just slightly related), help to fix vis to work with LuaJIT, particularly to make vis-lspc working.

---

MCP Server for SCC by digitaltomm

Description

Provide an MCP Server implementation for customers to access data on scc.suse.com via MCP protocol. Similar to the organization APIs, this can expose to customers data about their subscriptions, orders, systems and products. Authentication should be done by organization credentials, similar to what needs to be provided to RMT/MLM. Customers can connect to the SCC MCP server from their own MCP-compatible client and Large Language Model (LLM), so no third party is involved.

Goals

We want to demonstrate a proof of concept to connect to the SCC MCP server with any AI agent, like gemini-cli, copilot or Claude desktop. Enabling the user to ask questions regarding their SCC inventory, like "When do I need to re-new my SLES subscription", "Do I have active systems running on unsupported operating systems?".

Milestones

[ ] Basic MCP API setup
[ ] MCP endpoints
  [ ] Products / Repositories
  [ ] Subscriptions / Orders 
  [ ] Systems
[ ] Document usage with VSCode Copilot, Claude Desktop, Gemini CLI

Resources

• fast_mcp library for Rails
• MCP test client

---