RMT.rs: High-Performance Registration Path for RMT using Rust
Description
The SUSE Repository Mirroring Tool (RMT) is a critical component for managing software updates and subscriptions, especially for our Public Cloud Team (PCT). In a cloud environment, hundreds or even thousands of new SUSE instances (VPS/EC2) can be provisioned simultaneously. Each new instance attempts to register against an RMT server, creating a "thundering herd" scenario.
We have observed that the current RMT server, written in Ruby, faces performance issues under this high-concurrency registration load. This can lead to request overhead, slow registration times, and outright registration failures, delaying the readiness of new cloud instances.
This Hackweek project aims to explore a solution by re-implementing the performance-critical registration path in Rust. The goal is to leverage Rust's high performance, memory safety, and first-class concurrency handling to create an alternative registration endpoint that is fast, reliable, and can gracefully manage massive, simultaneous request spikes.
The new Rust module will be integrated into the existing RMT Ruby application, allowing us to directly compare the performance of both implementations.
Goals
The primary objective is to build and benchmark a high-performance Rust-based alternative for the RMT server registration endpoint.
Key goals for the week:
- Analyze & Identify: Dive into the
SUSE/rmtRuby codebase to identify and map out the exact critical path for server registration (e.g., controllers, services, database interactions). - Develop in Rust: Implement a functionally equivalent version of this registration logic in Rust.
- Integrate: Explore and implement a method for Ruby/Rust integration to "hot-wire" the new Rust module into the RMT application. This may involve using FFI, or libraries like
rb-sysormagnus. - Benchmark: Create a benchmarking script (e.g., using
k6,ab, or a custom tool) that simulates the high-concurrency registration load from thousands of clients. - Compare & Present: Conduct a comparative performance analysis (requests per second, latency, success/error rates, CPU/memory usage) between the original Ruby path and the new Rust path. The deliverable will be this data and a summary of the findings.
Resources
- RMT Source Code (Ruby):
https://github.com/SUSE/rmt
- RMT Documentation:
https://documentation.suse.com/sles/15-SP7/html/SLES-all/book-rmt.html
- Tooling & Stacks:
- RMT/Ruby development environment (for running the base RMT)
- Rust development environment (
rustup,cargo)
- Potential Integration Libraries:
- rb-sys:
https://github.com/oxidize-rb/rb-sys - Magnus:
https://github.com/matsadler/magnus
- rb-sys:
- Benchmarking Tools:
k6(https://k6.io/)ab(ApacheBench)
Looking for hackers with the skills:
rmt.rs rmt suse rust ruby performance optimization concurrency scalability thunderingherd registration ruby/rustintegration ffi benchmarking hackweek publiccloud
This project is part of:
Hack Week 25
Activity
Comments
Be the first to comment!
Similar Projects
Learn a bit of embedded programming with Rust in a micro:bit v2 by aplanas
Description
micro:bit is a small single board computer with a ARM Cortex-M4 with the FPU extension, with a very constrain amount of memory and a bunch of sensors and leds.
The board is very well documented, with schematics and code for all the features available, so is an excellent platform for learning embedded programming.
Rust is a system programming language that can generate ARM code, and has crates (libraries) to access the micro:bit hardware. There is plenty documentation about how to make small programs that will run in the micro:bit.
Goals
Start learning about embedded programming in Rust, and maybe make some code to the small KS4036F Robot car from keyestudio.
Resources
Diary
Day 1
- Start reading https://mb2.implrust.com/abstraction-layers.html
- Prepare the dev environment (cross compiler, probe-rs)
- Flash first code in the board (blinky led)
- Checking differences between BSP and HAL
- Compile and install a more complex example, with stack protection
- Reading about the simplicity of xtask, as alias for workspace execution
- Reading the CPP code of the official micro:bit libraries. They have a font!
Day 2
- There are multiple BSP for the microbit. One is using async code for non-blocking operations
- Download and study a bit the API for microbit-v2, the nRF official crate
- Take a look of the KS4036F programming, seems that the communication is multiplexed via I2C
- The motor speed can be selected via PWM (pulse with modulation): power it longer (high frequency), and it will increase the speed
- Scrolling some text
- Debug by printing! defmt is a crate that can be used with probe-rs to emit logs
- Start reading input from the board: buttons
- The logo can be touched and detected as a floating point value
Day 3
- A bit confused how to read the float value from a pin
- It is a typestate and I can only check islow / ishigh
- Move a bit into the HAL layer
- Ahh ... the floating is not about floating point value, is about driving or not (detecting) current
- So there are 5 timers of 32 bits. What's the frequency, Kenneth? (1MHz, 16MHz)
- Async in embedded Rust with Embassy!
- Using Embassy is a bit of cheating, so park it for now
- IRQs handlers for events triggered by devices
- Seems that SPI is for fast devices, and I2C for slow ones, like the accelerometer
- Start reading the rust-embedded book to see if concurrency should be done with interrupts
Build a terminal user-interface (TUI) for Agama by IGonzalezSosa
Description
Officially, Agama offers two different user interfaces. On the one hand, we have the web-based interface, which is the one you see when you run the installation media. On the other hand, we have a command-line interface. In both cases, you can use them using a remote system, either using a browser or the agama CLI.
We would expect most of the cases to be covered by this approach. However, if you cannot use the web-based interface and, for some reason, you cannot access the system through the network, your only option is to use the CLI. This interface offers a mechanism to modify Agama's configuration using an editor (vim, by default), but perhaps you might want to have a more user-friendly way.
Goals
The main goal of this project is to built a minimal terminal user-interface for Agama. This interface will allow the user to install the system providing just a few settings (selecting a product, a storage device and a user password). Then it should report the installation progress.
Resources
- https://agama-project.github.io/
- https://ratatui.rs/
AI-Powered Unit Test Automation for Agama by joseivanlopez
The Agama project is a multi-language Linux installer that leverages the distinct strengths of several key technologies:
- Rust: Used for the back-end services and the core HTTP API, providing performance and safety.
- TypeScript (React/PatternFly): Powers the modern web user interface (UI), ensuring a consistent and responsive user experience.
- Ruby: Integrates existing, robust YaST libraries (e.g.,
yast-storage-ng) to reuse established functionality.
The Problem: Testing Overhead
Developing and maintaining code across these three languages requires a significant, tedious effort in writing, reviewing, and updating unit tests for each component. This high cost of testing is a drain on developer resources and can slow down the project's evolution.
The Solution: AI-Driven Automation
This project aims to eliminate the manual overhead of unit testing by exploring and integrating AI-driven code generation tools. We will investigate how AI can:
- Automatically generate new unit tests as code is developed.
- Intelligently correct and update existing unit tests when the application code changes.
By automating this crucial but monotonous task, we can free developers to focus on feature implementation and significantly improve the speed and maintainability of the Agama codebase.
Goals
- Proof of Concept: Successfully integrate and demonstrate an authorized AI tool (e.g.,
gemini-cli) to automatically generate unit tests. - Workflow Integration: Define and document a new unit test automation workflow that seamlessly integrates the selected AI tool into the existing Agama development pipeline.
- Knowledge Sharing: Establish a set of best practices for using AI in code generation, sharing the learned expertise with the broader team.
Contribution & Resources
We are seeking contributors interested in AI-powered development and improving developer efficiency. Whether you have previous experience with code generation tools or are eager to learn, your participation is highly valuable.
If you want to dive deep into AI for software quality, please reach out and join the effort!
- Authorized AI Tools: Tools supported by SUSE (e.g.,
gemini-cli) - Focus Areas: Rust, TypeScript, and Ruby components within the Agama project.
Interesting Links
Exploring Rust's potential: from basics to security by sferracci
Description
This project aims to conduct a focused investigation and practical application of the Rust programming language, with a specific emphasis on its security model. A key component will be identifying and understanding the most common vulnerabilities that can be found in Rust code.
Goals
Achieve a beginner/intermediate level of proficiency in writing Rust code. This will be measured by trying to solve LeetCode problems focusing on common data structures and algorithms. Study Rust vulnerabilities and learning best practices to avoid them.
Resources
Rust book: https://doc.rust-lang.org/book/
Arcticwolf - A rust based user space NFS server by vcheng
Description
Rust has similar performance to C. Also, have a better async IO module and high integration with io_uring. This project aims to develop a user-space NFS server based on Rust.
Goals
- Get an understanding of how cargo works
- Get an understanding of how XDR was generated with xdrgen
- Create the RUST-based NFS server that supports basic operations like mount/readdir/read/write
Resources
https://github.com/Vicente-Cheng/arcticwolf
Recipes catalog and calculator in Rails 8 by gfilippetti
My wife needs a website to catalog and sell the products of her upcoming bakery, and I need to learn and practice modern Rails. So I'm using this Hack Week to build a modern store using the latest Ruby on Rails best practices, ideally up to the deployment.
TO DO
- Index page
- Product page
- Admin area -- Supplies calculator based on orders -- Orders notification
- Authentication
- Payment
- Deployment
Day 1
As my Rails knowledge was pretty outdated and I had 0 experience with Turbo (wich I want to use in the app), I started following a turbo-rails course. I completed 5 of 11 chapters.
Day 2
Continued the course until chapter 8 and added live updates & an empty state to the app. I should finish the course on day 3 and start my own project with the knowledge from it.
Hackweek 25
For this Hackweek I'll continue this project, focusing on a Catalog/Calculator for my wife's recipes so she can use for her Café.
Day 1
Improve the picotm Transaction Manager by tdz
Picotm is a system-level transaction manager. It provides transactional semantics to low-level C operations, such as
- memory access,
- modifying data structures,
- (some) file I/O, and
- common interfaces from the C Standard Library and POSIX.
Picotm also handles error detection and recovery for all it's functionality. It's fully modular, so new functionality can be added.
For the Hackweek, I want to dedicate some time to picotm. I want to finish some of the refactoring work that I have been working on. If there's time left, I'd like to investigate two-phase commits and how to support them in picotm.
Picotm is available at http://picotm.org/.