Explore LLM evaluation metrics by thbertoldi

Description

Learn the best practices for evaluating LLM performance with an open-source framework such as DeepEval.

Goals

Curate the knowledge learned during practice and present it to colleagues.

-> Maybe publish a blog post on SUSE's blog?

Resources

https://deepeval.com

https://docs.pactflow.io/docs/bi-directional-contract-testing

---

Liz - Prompt autocomplete by ftorchia

Description

Liz is the Rancher AI assistant for cluster operations.

Goals

We want to help users when sending new messages to Liz, by adding an autocomplete feature to complete their requests based on the context.

Example:

• User prompt: "Can you show me the list of p"
• Autocomplete suggestion: "Can you show me the list of p...od in local cluster?"

Example:

• User prompt: "Show me the logs of #rancher-"
• Chat console: It shows a drop-down widget, next to the # character, with the list of available pod names starting with "rancher-".

Technical Overview

1. The AI agent should expose a new ws/autocomplete endpoint to proxy autocomplete messages to the LLM.
2. The UI extension should be able to display prompt suggestions and allow users to apply the autocomplete to the Prompt via keyboard shortcuts.

Resources

GitHub repository

---

Uyuni Health-check Grafana AI Troubleshooter by ygutierrez

Description

This project explores the feasibility of using the open-source Grafana LLM plugin to enhance the Uyuni Health-check tool with LLM capabilities. The idea is to integrate a chat-based "AI Troubleshooter" directly into existing dashboards, allowing users to ask natural-language questions about errors, anomalies, or performance issues.

Goals

• Investigate if and how the grafana-llm-app plug-in can be used within the Uyuni Health-check tool.
• Investigate if this plug-in can be used to query LLMs for troubleshooting scenarios.
• Evaluate support for local LLMs and external APIs through the plugin.
• Evaluate if and how the Uyuni MCP server could be integrated as another source of information.

Resources

Grafana LMM plug-in

Uyuni Health-check

---

Multi-agent AI assistant for Linux troubleshooting by doreilly

Description

Explore multi-agent architecture as a way to avoid MCP context rot.

Having one agent with many tools bloats the context with low-level details about tool descriptions, parameter schemas etc which hurts LLM performance. Instead have many specialised agents, each with just the tools it needs for its role. A top level supervisor agent takes the user prompt and delegates to appropriate sub-agents.

Goals

Create an AI assistant with some sub-agents that are specialists at troubleshooting Linux subsystems, e.g. systemd, selinux, firewalld etc. The agents can get information from the system by implementing their own tools with simple function calls, or use tools from MCP servers, e.g. a systemd-agent can use tools from systemd-mcp.

Example prompts/responses:

user$ the system seems slow
assistant$ process foo with pid 12345 is using 1000% cpu ...

user$ I can't connect to the apache webserver
assistant$ the firewall is blocking http ... you can open the port with firewall-cmd --add-port ...

Resources

Language Python. The Python ADK is more mature than Golang.

https://google.github.io/adk-docs/

https://github.com/djoreilly/linux-helper

---

Update M2Crypto by mcepl

There are couple of projects I work on, which need my attention and putting them to shape:

• M2Crypto

Goal for this Hackweek

• Put M2Crypto into better shape (most issues closed, all pull requests processed)
• More fun to learn jujutsu
• Play more with Gemini, how much it help (or not).
• Perhaps, also (just slightly related), help to fix vis to work with LuaJIT, particularly to make vis-lspc working.

---

"what is it" file and directory analysis via MCP and local LLM, for console and KDE by rsimai

Description

Users sometimes wonder what files or directories they find on their local PC are good for. If they can't determine from the filename or metadata, there should an easy way to quickly analyze the content and at least guess the meaning. An LLM could help with that, through the use of a filesystem MCP and to-text-converters for typical file types. Ideally this is integrated into the desktop environment but works as well from a console. All data is processed locally or "on premise", no artifacts remain or leave the system.

Goals

• The user can run a command from the console, to check on a file or directory
• The filemanager contains the "analyze" feature within the context menu
• The local LLM could serve for other use cases where privacy matters

TBD

• Find or write capable one-shot and interactive MCP client
• Find or write simple+secure file access MCP server
• Create local LLM service with appropriate footprint, containerized
• Shell command with options
• KDE integration (Dolphin)
• Package
• Document

Resources

---

Bugzilla goes AI - Phase 1 by nwalter

Description

This project, Bugzilla goes AI, aims to boost developer productivity by creating an autonomous AI bug agent during Hackweek. The primary goal is to reduce the time employees spend triaging bugs by integrating Ollama to summarize issues, recommend next steps, and push focused daily reports to a Web Interface.

Goals

To reduce employee time spent on Bugzilla by implementing an AI tool that triages and summarizes bug reports, providing actionable recommendations to the team via Web Interface.

Project Charter

Bugzilla goes AI Phase 1

Description

Project Achievements during Hackweek

In this file you can read about what we achieved during Hackweek.

Project Achievements

---

Creating test suite using LLM on existing codebase of a solar router by fcrozat

Description

Two years ago, I evaluated solar routers as part of hackweek24, I've assembled one and it is running almost smoothly.

However, its code quality is not perfect and the codebase doesn't have any testcase (which is tricky, since it is embedded code and rely on getting external data to react).

Before improving the code itself, a testsuite should be created to ensure code additional don't cause regression.

Goals

Create a testsuite, allowing to test solar router code in a virtual environment. Using LLM to help to create this test suite.

If succesful, try to improve the codebase itself by having it reviewed by LLM.

Resources

Solar router github project

---

issuefs: FUSE filesystem representing issues (e.g. JIRA) for the use with AI agents code-assistants by llansky3

Description

Creating a FUSE filesystem (issuefs) that mounts issues from various ticketing systems (Github, Jira, Bugzilla, Redmine) as files to your local file system.

And why this is good idea?

• User can use favorite command line tools to view and search the tickets from various sources
• User can use AI agents capabilities from your favorite IDE or cli to ask question about the issues, project or functionality while providing relevant tickets as context without extra work.
• User can use it during development of the new features when you let the AI agent to jump start the solution. The issuefs will give the AI agent the context (AI agents just read few more files) about the bug or requested features. No need for copying and pasting issues to user prompt or by using extra MCP tools to access the issues. These you can still do but this approach is on purpose different.

Goals

1. Add Github issue support
2. Proof the concept/approach by apply the approach on itself using Github issues for tracking and development of new features
3. Add support for Bugzilla and Redmine using this approach in the process of doing it. Record a video of it.
4. Clean-up and test the implementation and create some documentation
5. Create a blog post about this approach

Resources

There is a prototype implementation here. This currently sort of works with JIRA only.

---

Extended private brain - RAG my own scripts and data into offline LLM AI by tjyrinki_suse

Description

For purely studying purposes, I'd like to find out if I could teach an LLM some of my own accumulated knowledge, to use it as a sort of extended brain.

I might use qwen3-coder or something similar as a starting point.

Everything would be done 100% offline without network available to the container, since I prefer to see when network is needed, and make it so it's never needed (other than initial downloads).

Goals

1. Learn something about RAG, LLM, AI.
2. Find out if everything works offline as intended.
3. As an end result have a new way to access my own existing know-how, but so that I can query the wisdom in them.
4. Be flexible to pivot in any direction, as long as there are new things learned.

Resources

To be found on the fly.

Timeline

Day 1 (of 4)

• Tried out a RAG demo, expanded on feeding it my own data
• Experimented with qwen3-coder to add a persistent chat functionality, and keeping vectors in a pickle file
• Optimizations to keep everything within context window
• Learn and add a bit of PyTest

Day 2

• More experimenting and more data
• Study ChromaDB
• Add a Web UI that works from another computer even though the container sees network is down

Day 3

• The above RAG is working well enough for demonstration purposes.
• Pivot to trying out OpenCode, configuring local Ollama qwen3-coder there, to analyze the RAG demo.
• Figured out how to configure Ollama template to be usable under OpenCode. OpenCode locally is super slow to just running qwen3-coder alone.

Day 4 (final day)

• Battle with OpenCode that was both slow and kept on piling up broken things.
• Call it success as after all the agentic AI was working locally.
• Clean up the mess left behind a bit.

Blog Post

Summarized the findings at blog post.

---

Kubernetes-Based ML Lifecycle Automation by lmiranda

Description

This project aims to build a complete end-to-end Machine Learning pipeline running entirely on Kubernetes, using Go, and containerized ML components.

The pipeline will automate the lifecycle of a machine learning model, including:

• Data ingestion/collection
• Model training as a Kubernetes Job
• Model artifact storage in an S3-compatible registry (e.g. Minio)
• A Go-based deployment controller that automatically deploys new model versions to Kubernetes using Rancher
• A lightweight inference service that loads and serves the latest model
• Monitoring of model performance and service health through Prometheus/Grafana

The outcome is a working prototype of an MLOps workflow that demonstrates how AI workloads can be trained, versioned, deployed, and monitored using the Kubernetes ecosystem.

Goals

By the end of Hack Week, the project should:

1. Produce a fully functional ML pipeline running on Kubernetes with:

   • Data collection job
   • Training job container
   • Storage and versioning of trained models
   • Automated deployment of new model versions
   • Model inference API service
   • Basic monitoring dashboards

2. Showcase a Go-based deployment automation component, which scans the model registry and automatically generates & applies Kubernetes manifests for new model versions.

3. Enable continuous improvement by making the system modular and extensible (e.g., additional models, metrics, autoscaling, or drift detection can be added later).

4. Prepare a short demo explaining the end-to-end process and how new models flow through the system.

Resources

Project Repository

Updates

1. Training pipeline and datasets
2. Inference Service py

---

MCP Server for SCC by digitaltomm

Description

Provide an MCP Server implementation for customers to access data on scc.suse.com via MCP protocol. The core benefit of this MCP interface is that it has direct (read) access to customer data in SCC, so the AI agent gets enhanced knowledge about individual customer data, like subscriptions, orders and registered systems.

Architecture

Goals

We want to demonstrate a proof of concept to connect to the SCC MCP server with any AI agent, for example gemini-cli or codex. Enabling the user to ask questions regarding their SCC inventory.

For this Hackweek, we target that users get proper responses to these example questions:

• Which of my currently active systems are running products that are out of support?
• Do I have ready to use registration codes for SLES?
• What are the latest 5 released patches for SLES 15 SP6? Output as a list with release date, patch name, affected package names and fixed CVEs.
• Which versions of kernel-default are available on SLES 15 SP6?

Technical Notes

Similar to the organization APIs, this can expose to customers data about their subscriptions, orders, systems and products. Authentication should be done by organization credentials, similar to what needs to be provided to RMT/MLM. Customers can connect to the SCC MCP server from their own MCP-compatible client and Large Language Model (LLM), so no third party is involved.

Milestones

[x] Basic MCP API setup
  MCP endpoints
  [x] Products / Repositories
  [x] Subscriptions / Orders 
  [x] Systems
  [x] Packages
[x] Document usage with Gemini CLI, Codex

Resources

• fast_mcp library for Rails
• MCP test client
• SCC branch

Gemini CLI setup:

~/.gemini/settings.json:

---