Local AI assistant with optional integrations and mobile companion by livdywan

Description

Setup a local AI assistant for research, brainstorming and proof reading. Look into SurfSense, Open WebUI and possibly alternatives. Explore integration with services like openQA. There should be no cloud dependencies. Mobile phone support or an additional companion app would be a bonus. The goal is not to develop everything from scratch.

User Story

• Allison Average wants a one-click local AI assistent on their openSUSE laptop.
• Ash Awesome wants AI on their phone without an expensive subscription.

Goals

• Evaluate a local SurfSense setup for day to day productivity
• Test opencode for vibe coding and tool calling

Timeline

Day 1

• Took a look at SurfSense and started setting up a local instance.
• Unfortunately the container setup did not work well. Tho this was a great opportunity to learn some new podman commands and refresh my memory on how to recover a corrupted btrfs filesystem.

Day 2

• Due to its sheer size and complexity SurfSense seems to have triggered btrfs fragmentation. Naturally this was not visible in any podman-related errors or in the journal. So this took up much of my second day.

Day 3

• Trying out opencode with Qwen3-Coder and Qwen2.5-Coder.

Day 4

• Context size is a thing, and models are not equally usable for vibe coding.
• Through arduous browsing for ollama models I did find some like myaniu/qwen2.5-1m:7b with 1m but even then it is not obvious if they are meant for tool calls.

Day 5

• Whilst trying to make opencode usable I discovered ramalama which worked instantly and very well.

Outcomes

surfsense

I could not easily set this up completely. Maybe in part due to my filesystem issues. Was expecting this to be less of an effort.

opencode

Installing opencode and ollama in my distrobox container along with the following configs worked for me.

When preparing a new project from scratch it is a good idea to start out with a template.

opencode.json

``` {

---

Try out Neovim Plugins supporting AI Providers by enavarro_suse

Description

Experiment with several Neovim plugins that integrate AI model providers such as Gemini and Ollama.

Goals

Evaluate how these plugins enhance the development workflow, how they differ in capabilities, and how smoothly they integrate into Neovim for day-to-day coding tasks.

Resources

• Neovim 0.11.5
• AI-enabled Neovim plugins:
  • avante.nvim: https://github.com/yetone/avante.nvim
  • Gp.nvim: https://github.com/Robitx/gp.nvim
  • parrot.nvim: https://github.com/frankroeder/parrot.nvim
  • gemini.nvim: https://dotfyle.com/plugins/kiddos/gemini.nvim
  • ...
• Accounts or API keys for AI model providers.
• Local model serving setup (e.g., Ollama)
• Test projects or codebases for practical evaluation:
  • OBS: https://build.opensuse.org/
  • OBS blog and landing page: https://openbuildservice.org/
  • ...

---

"what is it" file and directory analysis via MCP and local LLM, for console and KDE by rsimai

Description

Users sometimes wonder what files or directories they find on their local PC are good for. If they can't determine from the filename or metadata, there should an easy way to quickly analyze the content and at least guess the meaning. An LLM could help with that, through the use of a filesystem MCP and to-text-converters for typical file types. Ideally this is integrated into the desktop environment but works as well from a console. All data is processed locally or "on premise", no artifacts remain or leave the system.

Goals

• The user can run a command from the console, to check on a file or directory
• The filemanager contains the "analyze" feature within the context menu
• The local LLM could serve for other use cases where privacy matters

TBD

• Find or write capable one-shot and interactive MCP client
• Find or write simple+secure file access MCP server
• Create local LLM service with appropriate footprint, containerized
• Shell command with options
• KDE integration (Dolphin)
• Package
• Document

Resources

---

Try AI training with ROCm and LoRA by bmwiedemann

Description

I want to setup a Radeon RX 9600 XT 16 GB at home with ROCm on Slowroll.

Goals

I want to test how fast AI inference can get with the GPU and if I can use LoRA to re-train an existing free model for some task.

Resources

• https://rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html
• https://build.opensuse.org/project/show/science:GPU:ROCm
• https://src.opensuse.org/ROCm/
• https://www.suse.com/c/lora-fine-tuning-llms-for-text-classification/

Results

got inference working with llama.cpp:

export LLAMACPP_ROCM_ARCH=gfx1200
HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
cmake -S . -B build -DGGML_HIP=ON -DAMDGPU_TARGETS=$LLAMACPP_ROCM_ARCH \
-DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=ON \
-Dhipblas_DIR=/usr/lib64/cmake/hipblaslt/ \
&& cmake --build build --config Release -j8
m=models/gpt-oss-20b-mxfp4.gguf
cd $P/llama.cpp && build/bin/llama-server --model $m --threads 8 --port 8005 --host 0.0.0.0 --device ROCm0 --n-gpu-layers 999

Without the --device option it faulted. Maybe because my APU also appears there?

I updated/fixed various related packages: https://src.opensuse.org/ROCm/rocm-examples/pulls/1 https://src.opensuse.org/ROCm/hipblaslt/pulls/1 SR 1320959

benchmark

I benchmarked inference with llama.cpp + gpt-oss-20b-mxfp4.gguf and ROCm offloading to a Radeon RX 9060 XT 16GB. I varied the number of layers that went to the GPU:

• 0 layers 14.49 tokens/s (8 CPU cores)
• 9 layers 17.79 tokens/s 34% VRAM
• 15 layers 22.39 tokens/s 51% VRAM
• 20 layers 27.49 tokens/s 64% VRAM
• 24 layers 41.18 tokens/s 74% VRAM
• 25+ layers 86.63 tokens/s 75% VRAM (only 200% CPU load)

So there is a significant performance-boost if the whole model fits into the GPU's VRAM.

---

Self-Scaling LLM Infrastructure Powered by Rancher by ademicev0

Self-Scaling LLM Infrastructure Powered by Rancher

---

Description

The Problem

Running LLMs can get expensive and complex pretty quickly.

Today there are typically two choices:

1. Use cloud APIs like OpenAI or Anthropic. Easy to start with, but costs add up at scale.
2. Self-host everything - set up Kubernetes, figure out GPU scheduling, handle scaling, manage model serving... it's a lot of work.

What if there was a middle ground?

What if infrastructure scaled itself instead of making you scale it?

Can we use existing Rancher capabilities like CAPI, autoscaling, and GitOps to make this simpler instead of building everything from scratch?

Project Repository: github.com/alexander-demicev/llmserverless

---

What This Project Does

A key feature is hybrid deployment: requests can be routed based on complexity or privacy needs. Simple or low-sensitivity queries can use public APIs (like OpenAI), while complex or private requests are handled in-house on local infrastructure. This flexibility allows balancing cost, privacy, and performance - using cloud for routine tasks and on-premises resources for sensitive or demanding workloads.

A complete, self-scaling LLM infrastructure that:

• Scales to zero when idle (no idle costs)
• Scales up automatically when requests come in
• Adds more nodes when needed, removes them when demand drops
• Runs on any infrastructure - laptop, bare metal, or cloud

Think of it as "serverless for LLMs" - focus on building, the infrastructure handles itself.

How It Works

A combination of open source tools working together:

Flow:

• Users interact with OpenWebUI (chat interface)
• Requests go to LiteLLM Gateway
• LiteLLM routes requests to:
  • Ollama (Knative) for local model inference (auto-scales pods)
  • Or cloud APIs for fallback

---

Exploring Modern AI Trends and Kubernetes-Based AI Infrastructure by jluo

Description

Build a solid understanding of the current landscape of Artificial Intelligence and how modern cloud-native technologies—especially Kubernetes—support AI workloads.

Goals

Use Gemini Learning Mode to guide the exploration, surface relevant concepts, and structure the learning journey:

• Gain insight into the latest AI trends, tools, and architectural concepts.
• Understand how Kubernetes and related cloud-native technologies are used in the AI ecosystem (model training, deployment, orchestration, MLOps).

Resources

• Red Hat AI Topic Articles

  • https://www.redhat.com/en/topics/ai

• Kubeflow Documentation

  • https://www.kubeflow.org/docs/

• Q4 2025 CNCF Technology Landscape Radar report:

  • https://www.cncf.io/announcements/2025/11/11/cncf-and-slashdata-report-finds-leading-ai-tools-gaining-adoption-in-cloud-native-ecosystems/
  • https://www.cncf.io/wp-content/uploads/2025/11/cncfreporttechradar_111025a.pdf

• Agent-to-Agent (A2A) Protocol

  • https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/

---

Kubernetes-Based ML Lifecycle Automation by lmiranda

Description

This project aims to build a complete end-to-end Machine Learning pipeline running entirely on Kubernetes, using Go, and containerized ML components.

The pipeline will automate the lifecycle of a machine learning model, including:

• Data ingestion/collection
• Model training as a Kubernetes Job
• Model artifact storage in an S3-compatible registry (e.g. Minio)
• A Go-based deployment controller that automatically deploys new model versions to Kubernetes using Rancher
• A lightweight inference service that loads and serves the latest model
• Monitoring of model performance and service health through Prometheus/Grafana

The outcome is a working prototype of an MLOps workflow that demonstrates how AI workloads can be trained, versioned, deployed, and monitored using the Kubernetes ecosystem.

Goals

By the end of Hack Week, the project should:

1. Produce a fully functional ML pipeline running on Kubernetes with:

   • Data collection job
   • Training job container
   • Storage and versioning of trained models
   • Automated deployment of new model versions
   • Model inference API service
   • Basic monitoring dashboards

2. Showcase a Go-based deployment automation component, which scans the model registry and automatically generates & applies Kubernetes manifests for new model versions.

3. Enable continuous improvement by making the system modular and extensible (e.g., additional models, metrics, autoscaling, or drift detection can be added later).

4. Prepare a short demo explaining the end-to-end process and how new models flow through the system.

Resources

Project Repository

Updates

1. Training pipeline and datasets
2. Inference Service py

---

Create a Cloud-Native policy engine with notifying capabilities to optimize resource usage by gbazzotti

Description

The goal of this project is to begin the initial phase of development of an all-in-one Cloud-Native Policy Engine that notifies resource owners when their resources infringe predetermined policies. This was inspired by a current issue in the CES-SRE Team where other solutions seemed to not exactly correspond to the needs of the specific workloads running on the Public Cloud Team space.

The initial architecture can be checked out on the Repository listed under Resources.

Among the features that will differ this project from other monitoring/notification systems:

• Pre-defined sensible policies written at the software-level, avoiding a learning curve by requiring users to write their own policies
• All-in-one functionality: logging, mailing and all other actions are not required to install any additional plugins/packages
• Easy account management, being able to parse all required configuration by a single JSON file
• Eliminate integrations by not requiring metrics to go through a data-agreggator

Goals

• Create a minimal working prototype following the workflow specified on the documentation
• Provide instructions on installation/usage
• Work on email notifying capabilities

Resources

• GitHub Repository

---