Description

Build a solid understanding of the current landscape of Artificial Intelligence and how modern cloud-native technologies—especially Kubernetes—support AI workloads.

Goals

Use Gemini Learning Mode to guide the exploration, surface relevant concepts, and structure the learning journey:

Gain insight into the latest AI trends, tools, and architectural concepts.
Understand how Kubernetes and related cloud-native technologies are used in the AI ecosystem (model training, deployment, orchestration, MLOps).

Resources

Red Hat AI Topic Articles
- https://www.redhat.com/en/topics/ai
Kubeflow Documentation
- https://www.kubeflow.org/docs/
Q4 2025 CNCF Technology Landscape Radar report:
- https://www.cncf.io/announcements/2025/11/11/cncf-and-slashdata-report-finds-leading-ai-tools-gaining-adoption-in-cloud-native-ecosystems/
- https://www.cncf.io/wp-content/uploads/2025/11/cncfreporttechradar_111025a.pdf
Agent-to-Agent (A2A) Protocol
- https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/

Looking for hackers with the skills:

ai aiops kubernetes mlops

This project is part of:

Hack Week 25

Activity

19 days ago: horon disliked this project.

19 days ago: horon liked this project.

20 days ago: jluo added keyword "ai" to this project.

20 days ago: jluo added keyword "aiops" to this project.

20 days ago: jluo added keyword "kubernetes" to this project.

20 days ago: jluo added keyword "mlops" to this project.

20 days ago: jluo started this project.

20 days ago: jluo originated this project.

Comments

20 days ago by jluo | Reply

A great summary from Gemini:

This is a rapidly expanding ecosystem. To keep it organized, I will break this list down by Lifecycle Stage (Training vs. Serving) and Infrastructure Layer (Compute vs. Data).

1. AI Platforms & Orchestration (The "Command Center")

These tools manage the end-to-end lifecycle, gluing everything else together.
- Kubeflow: The "Grandfather" of AI on K8s. It’s a massive suite including:
  - Kubeflow Pipelines: For building repeatable workflows (Data -> Train -> Deploy).
  - Kubeflow Notebooks: Spawns Jupyter servers as Pods for data scientists.
  - Katib: Automated hyperparameter tuning (finding the best learning rate).
- Ray (KubeRay): The top challenger to Kubeflow. It allows you to write Python code that scales across a cluster instantly. It is excellent for both distributed training and serving.
- ZenML: An MLOps framework that sits above the infrastructure, letting you define pipelines in code that can run on Kubeflow, Ray, or simple Kubernetes batches.
2. Training & Scheduling (The "Heavy Lifters")

Standard Kubernetes scheduling (FIFO) is bad for AI training. These tools fix that.
- Volcano: A batch scheduler. It ensures "Gang Scheduling"—meaning if a job needs 50 GPUs but only 49 are available, it waits. (Standard K8s would start 49 and let them sit idle, wasting money).
- Kueue: A newer, lighter alternative to Volcano managed by the K8s specialized interest group. It manages "Job Queues" natively.
- Training Operator: A unified K8s operator that lets you run PyTorchJob, TFJob (TensorFlow), and XGBoostJob as native K8s objects.
3. Inference & Serving (The "Waiter")

Once a model is trained, these tools serve it to users.
- KServe: The industry standard. It handles "Scale-to-Zero" (via KEDA), canary rollouts, and provides a unified API for TensorFlow, PyTorch, and ONNX models.
- vLLM: The current king of LLM serving. It is highly optimized for GPU memory (PagedAttention) and is often run inside KServe or as a standalone Deployment.
- BentoML / Yatai: A developer-friendly framework. You package your model as a "Bento" (standard format), and Yatai orchestrates the deployment on K8s.
- Seldon Core: An enterprise-grade alternative to KServe with advanced features for compliance, audit trails, and complex inference graphs.
4. Agentic & LLM Ops (The "New Wave")

Tools specifically for the 2025 era of Autonomous Agents.
- LangFlow / Flowise: Low-code "drag-and-drop" UI tools for building LLM chains. They can be deployed on K8s via Helm charts to run agent backends.
- kagent / Agent Sandbox: Emerging tools (often cloud-specific or experimental) that provide secure, isolated environments (using gVisor or microVMs) for agents to execute code safely.
- Ollama: While often used locally, it is increasingly deployed on K8s (via Helm) as a lightweight way to serve open-source models like Llama 3 or Mistral inside a cluster.
5. Data & Memory (The "Brain")
- Vector Databases (with K8s Operators):
  - Milvus: A popular open-source vector DB built natively for K8s scalability.
  - Weaviate: Another strong option with a solid K8s operator.
  - Qdrant: Written in Rust, very fast, and easy to deploy on K8s.
- Feature Stores:
  - Feast: The open-source standard for serving features (e.g., "User's last 5 clicks") to models in real-time.
6. Observability & Cost (The "Watchtower")
- Prometheus & Grafana: The standard for metrics (GPU temperature, Request Latency).
- DCGM Exporter: The specific NVIDIA tool that pulls GPU metrics (utilization, memory) so Prometheus can see them.
- KEDA: The autoscaler (discussed previously) that scales pods based on event queues.
- Karpenter: The Node autoscaler. If KEDA asks for more pods, Karpenter instantly buys more EC2/VM nodes from the cloud provider to fit them.
- OpenCost / Kubecost: Tools to track exactly how much money your AI team is spending on GPUs per namespace.

20 days ago by jluo | Reply

Interact with GitHub Copilot to evaluate its capability for assisting with daily work.

For the result PR, see https://github.com/rancher/rancher/pull/52943

Similar Projects

ai

SUSE Observability MCP server by drutigliano

Description

The idea is to implement the SUSE Observability Model Context Protocol (MCP) Server as a specialized, middle-tier API designed to translate the complex, high-cardinality observability data from StackState (topology, metrics, and events) into highly structured, contextually rich, and LLM-ready snippets.

This MCP Server abstract the StackState APIs. Its primary function is to serve as a Tool/Function Calling target for AI agents. When an AI receives an alert or a user query (e.g., "What caused the outage?"), the AI calls an MCP Server endpoint. The server then fetches the relevant operational facts, summarizes them, normalizes technical identifiers (like URNs and raw metric names) into natural language concepts, and returns a concise JSON or YAML payload. This payload is then injected directly into the LLM's prompt, ensuring the final diagnosis or action is grounded in real-time, accurate SUSE Observability data, effectively minimizing hallucinations.

Goals

Grounding AI Responses: Ensure that all AI diagnoses, root cause analyses, and action recommendations are strictly based on verifiable, real-time data retrieved from the SUSE Observability StackState platform.
Simplifying Data Access: Abstract the complexity of StackState's native APIs (e.g., Time Travel, 4T Data Model) into simple, semantic functions that can be easily invoked by LLM tool-calling mechanisms.
Data Normalization: Convert complex, technical identifiers (like component URNs, raw metric names, and proprietary health states) into standardized, natural language terms that an LLM can easily reason over.
Enabling Automated Remediation: Define clear, action-oriented MCP endpoints (e.g., execute_runbook) that allow the AI agent to initiate automated operational workflows (e.g., restarts, scaling) after a diagnosis, closing the loop on observability.

Hackweek STEP

Create a functional MCP endpoint exposing one (or more) tool(s) to answer queries like "What is the health of service X?") by fetching, normalizing, and returning live StackState data in an LLM-ready format.

Scope

Implement read-only MCP server that can:
- Connect to a live SUSE Observability instance and authenticate (with API token)
- Use tools to fetch data for a specific component URN (e.g., current health state, metrics, possibly topology neighbors, ...).
- Normalize response fields (e.g., URN to "Service Name," health state DEVIATING to "Unhealthy", raw metrics).
- Return the data as a structured JSON payload compliant with the MCP specification.

Deliverables

MCP Server v0.1 A running Golang MCP server with at least one tool.
A README.md and a test script (e.g., curl commands or a simple notebook) showing how an AI agent would call the endpoint and the resulting JSON payload.

Outcome A functional and testable API endpoint that proves the core concept: translating complex StackState data into a simple, LLM-ready format. This provides the foundation for developing AI-driven diagnostics and automated remediation.

Resources

https://www.honeycomb.io/blog/its-the-end-of-observability-as-we-know-it-and-i-feel-fine
https://www.datadoghq.com/blog/datadog-remote-mcp-server
https://modelcontextprotocol.io/specification/2025-06-18/index
https://modelcontextprotocol.io/docs/develop/build-server

Basic implementation

https://github.com/drutigliano19/suse-observability-mcp-server

Results

Successfully developed and delivered a fully functional SUSE Observability MCP Server that bridges language models with SUSE Observability's operational data. This project demonstrates how AI agents can perform intelligent troubleshooting and root cause analysis using structured access to real-time infrastructure data.

Example execution

Multi-agent AI assistant for Linux troubleshooting by doreilly

Description

Explore multi-agent architecture as a way to avoid MCP context rot.

Having one agent with many tools bloats the context with low-level details about tool descriptions, parameter schemas etc which hurts LLM performance. Instead have many specialised agents, each with just the tools it needs for its role. A top level supervisor agent takes the user prompt and delegates to appropriate sub-agents.

Goals

Create an AI assistant with some sub-agents that are specialists at troubleshooting Linux subsystems, e.g. systemd, selinux, firewalld etc. The agents can get information from the system by implementing their own tools with simple function calls, or use tools from MCP servers, e.g. a systemd-agent can use tools from systemd-mcp.

Example prompts/responses:

user$ the system seems slow
assistant$ process foo with pid 12345 is using 1000% cpu ...

user$ I can't connect to the apache webserver
assistant$ the firewall is blocking http ... you can open the port with firewall-cmd --add-port ...

Resources

Language Python. The Python ADK is more mature than Golang.

https://google.github.io/adk-docs/

https://github.com/djoreilly/linux-helper

Liz - Prompt autocomplete by ftorchia

Description

Liz is the Rancher AI assistant for cluster operations.

Goals

We want to help users when sending new messages to Liz, by adding an autocomplete feature to complete their requests based on the context.

Example:

User prompt: "Can you show me the list of p"
Autocomplete suggestion: "Can you show me the list of p...od in local cluster?"

Example:

User prompt: "Show me the logs of #rancher-"
Chat console: It shows a drop-down widget, next to the # character, with the list of available pod names starting with "rancher-".

Technical Overview

The AI agent should expose a new ws/autocomplete endpoint to proxy autocomplete messages to the LLM.
The UI extension should be able to display prompt suggestions and allow users to apply the autocomplete to the Prompt via keyboard shortcuts.

Resources

GitHub repository

Enable more features in mcp-server-uyuni by j_renner

Description

I would like to contribute to mcp-server-uyuni, the MCP server for Uyuni / Multi-Linux Manager) exposing additional features as tools. There is lots of relevant features to be found throughout the API, for example:

At the end of the week I managed to enable basic system group operations:

List all system groups visible to the user
Create new system groups
List systems assigned to a group
Add and remove systems from groups

Goals

Set up test environment locally with the MCP server and client + a recent MLM server [DONE]
Identify features and use cases offering a benefit with limited effort required for enablement [DONE]
Create a PR to the repo [DONE]

Resources

Docs Navigator MCP: SUSE Edition by mackenzie.techdocs

Description

Docs Navigator MCP: SUSE Edition is an AI-powered documentation navigator that makes finding information across SUSE, Rancher, K3s, and RKE2 documentation effortless. Built as a Model Context Protocol (MCP) server, it enables semantic search, intelligent Q&A, and documentation summarization using 100% open-source AI models (no API keys required!). The project also allows you to bring your own keys from Anthropic and Open AI for parallel processing.

Goals

[ X ] Build functional MCP server with documentation tools
[ X ] Implement semantic search with vector embeddings
[ X ] Create user-friendly web interface
[ X ] Optimize indexing performance (parallel processing)
[ X ] Add SUSE branding and polish UX
[ X ] Stretch Goal: Add more documentation sources
[ X ] Stretch Goal: Implement document change detection for auto-updates

Coming Soon!

Community Feedback: Test with real users and gather improvement suggestions

Resources

Repository: Docs Navigator MCP: SUSE Edition GitHub
UI Demo: Live UI Demo of Docs Navigator MCP: SUSE Edition

aiops

Bugzilla goes AI - Phase 1 by nwalter

Description

This project, Bugzilla goes AI, aims to boost developer productivity by creating an autonomous AI bug agent during Hackweek. The primary goal is to reduce the time employees spend triaging bugs by integrating Ollama to summarize issues, recommend next steps, and push focused daily reports to a Web Interface.

Goals

To reduce employee time spent on Bugzilla by implementing an AI tool that triages and summarizes bug reports, providing actionable recommendations to the team via Web Interface.

Project Charter

Bugzilla goes AI Phase 1

Description

Project Achievements during Hackweek

In this file you can read about what we achieved during Hackweek.

Project Achievements

SUSE Observability MCP server by drutigliano

Description

Goals

Grounding AI Responses: Ensure that all AI diagnoses, root cause analyses, and action recommendations are strictly based on verifiable, real-time data retrieved from the SUSE Observability StackState platform.
Simplifying Data Access: Abstract the complexity of StackState's native APIs (e.g., Time Travel, 4T Data Model) into simple, semantic functions that can be easily invoked by LLM tool-calling mechanisms.
Data Normalization: Convert complex, technical identifiers (like component URNs, raw metric names, and proprietary health states) into standardized, natural language terms that an LLM can easily reason over.
Enabling Automated Remediation: Define clear, action-oriented MCP endpoints (e.g., execute_runbook) that allow the AI agent to initiate automated operational workflows (e.g., restarts, scaling) after a diagnosis, closing the loop on observability.

Hackweek STEP

Create a functional MCP endpoint exposing one (or more) tool(s) to answer queries like "What is the health of service X?") by fetching, normalizing, and returning live StackState data in an LLM-ready format.

Scope

Implement read-only MCP server that can:
- Connect to a live SUSE Observability instance and authenticate (with API token)
- Use tools to fetch data for a specific component URN (e.g., current health state, metrics, possibly topology neighbors, ...).
- Normalize response fields (e.g., URN to "Service Name," health state DEVIATING to "Unhealthy", raw metrics).
- Return the data as a structured JSON payload compliant with the MCP specification.

Deliverables

MCP Server v0.1 A running Golang MCP server with at least one tool.
A README.md and a test script (e.g., curl commands or a simple notebook) showing how an AI agent would call the endpoint and the resulting JSON payload.

Resources

https://www.honeycomb.io/blog/its-the-end-of-observability-as-we-know-it-and-i-feel-fine
https://www.datadoghq.com/blog/datadog-remote-mcp-server
https://modelcontextprotocol.io/specification/2025-06-18/index
https://modelcontextprotocol.io/docs/develop/build-server

Basic implementation

https://github.com/drutigliano19/suse-observability-mcp-server

Results

Example execution

Explore LLM evaluation metrics by thbertoldi

Description

Learn the best practices for evaluating LLM performance with an open-source framework such as DeepEval.

Goals

Curate the knowledge learned during practice and present it to colleagues.

-> Maybe publish a blog post on SUSE's blog?

Resources

https://deepeval.com

https://docs.pactflow.io/docs/bi-directional-contract-testing

kubernetes

Preparing KubeVirtBMC for project transfer to the KubeVirt organization by zchang

Description

KubeVirtBMC is preparing to transfer the project to the KubeVirt organization. One requirement is to enhance the modeling design's security. The current v1alpha1 API (the VirtualMachineBMC CRD) was designed during the proof-of-concept stage. It's immature and inherently insecure due to its cross-namespace object references, exposing security concerns from an RBAC perspective.

The other long-awaited feature is the ability to mount virtual media so that virtual machines can boot from remote ISO images.

Goals

Deliver the v1beta1 API and its corresponding controller implementation
Enable the Redfish virtual media mount function for KubeVirt virtual machines

Resources

A CLI for Harvester by mohamed.belgaied

Harvester does not officially come with a CLI tool, the user is supposed to interact with Harvester mostly through the UI. Though it is theoretically possible to use kubectl to interact with Harvester, the manipulation of Kubevirt YAML objects is absolutely not user friendly. Inspired by tools like multipass from Canonical to easily and rapidly create one of multiple VMs, I began the development of Harvester CLI. Currently, it works but Harvester CLI needs some love to be up-to-date with Harvester v1.0.2 and needs some bug fixes and improvements as well.

Project Description

Harvester CLI is a command line interface tool written in Go, designed to simplify interfacing with a Harvester cluster as a user. It is especially useful for testing purposes as you can easily and rapidly create VMs in Harvester by providing a simple command such as: harvester vm create my-vm --count 5 to create 5 VMs named my-vm-01 to my-vm-05.

Harvester CLI is functional but needs a number of improvements: up-to-date functionality with Harvester v1.0.2 (some minor issues right now), modifying the default behaviour to create an opensuse VM instead of an ubuntu VM, solve some bugs, etc.

Github Repo for Harvester CLI: https://github.com/belgaied2/harvester-cli

Done in previous Hackweeks

Create a Github actions pipeline to automatically integrate Harvester CLI to Homebrew repositories: DONE
Automatically package Harvester CLI for OpenSUSE / Redhat RPMs or DEBs: DONE

Goal for this Hackweek

The goal for this Hackweek is to bring Harvester CLI up-to-speed with latest Harvester versions (v1.3.X and v1.4.X), and improve the code quality as well as implement some simple features and bug fixes.

Some nice additions might be: * Improve handling of namespaced objects * Add features, such as network management or Load Balancer creation ? * Add more unit tests and, why not, e2e tests * Improve CI * Improve the overall code quality * Test the program and create issues for it

Issue list is here: https://github.com/belgaied2/harvester-cli/issues

Resources

The project is written in Go, and using client-go the Kubernetes Go Client libraries to communicate with the Harvester API (which is Kubernetes in fact). Welcome contributions are:

Testing it and creating issues
Documentation
Go code improvement

What you might learn

Harvester CLI might be interesting to you if you want to learn more about:

GitHub Actions
Harvester as a SUSE Product
Go programming language
Kubernetes API
Kubevirt API objects (Manipulating VMs and VM Configuration in Kubernetes using Kubevirt)

OpenPlatform Self-Service Portal by tmuntan1

Description

In SUSE IT, we developed an internal developer platform for our engineers using SUSE technologies such as RKE2, SUSE Virtualization, and Rancher. While it works well for our existing users, the onboarding process could be better.

To improve our customer experience, I would like to build a self-service portal to make it easy for people to accomplish common actions. To get started, I would have the portal create Jira SD tickets for our customers to have better information in our tickets, but eventually I want to add automation to reduce our workload.

Goals

Build a frontend website (Angular) that helps customers create Jira SD tickets.
Build a backend (Rust with Axum) for the backend, which would do all the hard work for the frontend.

Resources (SUSE VPN only)

development site: https://ui-dev.openplatform.suse.com/login?returnUrl=%2Fopenplatform%2Fforms
https://gitlab.suse.de/itpe/core/open-platform/op-portal/backend
https://gitlab.suse.de/itpe/core/open-platform/op-portal/frontend

Self-Scaling LLM Infrastructure Powered by Rancher by ademicev0

Self-Scaling LLM Infrastructure Powered by Rancher

Description

The Problem

Running LLMs can get expensive and complex pretty quickly.

Today there are typically two choices:

Use cloud APIs like OpenAI or Anthropic. Easy to start with, but costs add up at scale.
Self-host everything - set up Kubernetes, figure out GPU scheduling, handle scaling, manage model serving... it's a lot of work.

What if there was a middle ground?

What if infrastructure scaled itself instead of making you scale it?

Can we use existing Rancher capabilities like CAPI, autoscaling, and GitOps to make this simpler instead of building everything from scratch?

Project Repository: github.com/alexander-demicev/llmserverless

What This Project Does

A key feature is hybrid deployment: requests can be routed based on complexity or privacy needs. Simple or low-sensitivity queries can use public APIs (like OpenAI), while complex or private requests are handled in-house on local infrastructure. This flexibility allows balancing cost, privacy, and performance - using cloud for routine tasks and on-premises resources for sensitive or demanding workloads.

A complete, self-scaling LLM infrastructure that:

Scales to zero when idle (no idle costs)
Scales up automatically when requests come in
Adds more nodes when needed, removes them when demand drops
Runs on any infrastructure - laptop, bare metal, or cloud

Think of it as "serverless for LLMs" - focus on building, the infrastructure handles itself.

How It Works

A combination of open source tools working together:

Flow:

Users interact with OpenWebUI (chat interface)
Requests go to LiteLLM Gateway
LiteLLM routes requests to:
- Ollama (Knative) for local model inference (auto-scales pods)
- Or cloud APIs for fallback

Rancher/k8s Trouble-Maker by tonyhansen

Project Description

When studying for my RHCSA, I found trouble-maker, which is a program that breaks a Linux OS and requires you to fix it. I want to create something similar for Rancher/k8s that can allow for troubleshooting an unknown environment.

Goals for Hackweek 25

Update to modern Rancher and verify that existing tests still work
Change testing logic to populate secrets instead of requiring a secondary script
Add new tests

Goals for Hackweek 24 (Complete)

Create a basic framework for creating Rancher/k8s cluster lab environments as needed for the Break/Fix
Create at least 5 modules that can be applied to the cluster and require troubleshooting

Resources

https://github.com/celidon/rancher-troublemaker
https://github.com/rancher/terraform-provider-rancher2
https://github.com/rancher/tf-rancher-up
https://github.com/rancher/quickstart

mlops

Kubernetes-Based ML Lifecycle Automation by lmiranda

Description

This project aims to build a complete end-to-end Machine Learning pipeline running entirely on Kubernetes, using Go, and containerized ML components.

The pipeline will automate the lifecycle of a machine learning model, including:

Data ingestion/collection
Model training as a Kubernetes Job
Model artifact storage in an S3-compatible registry (e.g. Minio)
A Go-based deployment controller that automatically deploys new model versions to Kubernetes using Rancher
A lightweight inference service that loads and serves the latest model
Monitoring of model performance and service health through Prometheus/Grafana

The outcome is a working prototype of an MLOps workflow that demonstrates how AI workloads can be trained, versioned, deployed, and monitored using the Kubernetes ecosystem.

Goals

By the end of Hack Week, the project should:

Produce a fully functional ML pipeline running on Kubernetes with:
- Data collection job
- Training job container
- Storage and versioning of trained models
- Automated deployment of new model versions
- Model inference API service
- Basic monitoring dashboards
Showcase a Go-based deployment automation component, which scans the model registry and automatically generates & applies Kubernetes manifests for new model versions.
Enable continuous improvement by making the system modular and extensible (e.g., additional models, metrics, autoscaling, or drift detection can be added later).
Prepare a short demo explaining the end-to-end process and how new models flow through the system.

Resources

Project Repository

Updates

Training pipeline and datasets
Inference Service py

Description

Goals

Resources

Looking for hackers with the skills:

This project is part of:

Activity

Comments

20 days ago by jluo | Reply

1. AI Platforms & Orchestration (The "Command Center")

2. Training & Scheduling (The "Heavy Lifters")

3. Inference & Serving (The "Waiter")

4. Agentic & LLM Ops (The "New Wave")

5. Data & Memory (The "Brain")

6. Observability & Cost (The "Watchtower")

20 days ago by jluo | Reply

Similar Projects

ai

SUSE Observability MCP server by drutigliano

Description

Goals

Hackweek STEP

Scope

Deliverables

Resources

Basic implementation

Results

Example execution

Multi-agent AI assistant for Linux troubleshooting by doreilly

Description

Goals

Resources

Liz - Prompt autocomplete by ftorchia

Description

Goals

Technical Overview

Resources

Enable more features in mcp-server-uyuni by j_renner

Description

Goals

Resources

Docs Navigator MCP: SUSE Edition by mackenzie.techdocs

Description

Goals

Coming Soon!

Resources

aiops

Bugzilla goes AI - Phase 1 by nwalter

Description

Goals

Project Charter

Description

Project Achievements during Hackweek

SUSE Observability MCP server by drutigliano

Description

Goals

Hackweek STEP

Scope

Deliverables

Resources

Basic implementation

Results

Example execution

Explore LLM evaluation metrics by thbertoldi

Description

Goals

Resources

kubernetes

Preparing KubeVirtBMC for project transfer to the KubeVirt organization by zchang

A CLI for Harvester by mohamed.belgaied

Project Description

Done in previous Hackweeks

Goal for this Hackweek

Resources

What you might learn

OpenPlatform Self-Service Portal by tmuntan1

Description

Goals

Resources (SUSE VPN only)

Self-Scaling LLM Infrastructure Powered by Rancher by ademicev0

Self-Scaling LLM Infrastructure Powered by Rancher