Twopence is (will be) a remote execution engine for tests, able to run tests in virtual machines and real hardware through various means of communication : virtio for KVM / QEmu, ssh on top of libssh, serial lines. This library can be called from shell and ruby wrappers.
While it is already functional (and used), it still needs polishing, stabilizing, and extending. It is also planned to integrate it with Pennyworth (project Machinery) and let it go fully Open Source.
All crazy ideas and good will are welcome. It could be Python wrappers, support for multiple simultaneous tests, support for Linux containers, tests through a firewire cable, or whatever crosses your mind.
What was achieved
- - GitHub: The project moved to GitHub and is willing to play the rules of OpenSource;
- - Ctrl-C and other aborts: now it is possible to interrupt a remote execution with Ctrl-C, both from shell and from Ruby interface;
- - Pennyworth integration: now Pennyworth (project Machinery) calls Twopence to execute remote commands, inject or extract files.
What was not achieved
- - Bug: running the Twopence server for virtio or serial lines with output redirected to a file results in the whole output being repeated with each new message (probably some buffer not flushed).
- - Better packaging: we would need at very least separate packages for client and server parts;
- - Run over firewire: this item is a bit crazy - the idea is to use firewire to run tests and transfer files without using the network stack, by using either raw libraries or raw devices. I bought the necessary hardware but had no time to play with it :-( ;
- - Multiplexing: enable simultaneous and independent tests running on the same machine;
- - Ruby documentation: the Ruby API is currently undocumented.
What could be done better
- - Ctrl-C and other aborts: enable aborting a file transfer, not only remote execution of commands;
- - Pennyworth integration: transferring recursively a whole directory's contents is still done with Cheetah.
This project is part of:
Hack Week 11
Activity
Comments
-
-
about 11 years ago by e_bischoff | Reply
Currently only on https://build.suse.de/package/show/home:ebischoff/twopence (sources available). I was planning to move them to GitHub right for the hackweek.
-
about 11 years ago by jreidinger | Reply
I think it make good sense to also support VNC as a way how to communicate with machine as it immediatelly increase range where it is useful. BTW some comparison to openQA if it is orthogonal or if it can benefit from some synerge would be nice.
-
about 11 years ago by e_bischoff | Reply
Twopence is used to execute remote commands and transfer files just like you would do with scp or ssh (but works even with no network, unlike ssh). It is not a remote display system like VNC. It works only with a running system, unlike OpenQA, because it either needs a SSH server or a Twopence server on the System Under Tests, so it cannot test the early stages of boot. In short, there is not much duplication of functionality - at least with those two pieces of software.
-
over 9 years ago by e_bischoff | Reply
I'll mark this project as finished. Not everything has been done, but it is now stable and regularly used software.
Similar Projects
Improve the picotm Transaction Manager by tdz
Picotm is a system-level transaction manager. It provides transactional semantics to low-level C operations, such as
- memory access,
- modifying data structures,
- (some) file I/O, and
- common interfaces from the C Standard Library and POSIX.
Picotm also handles error detection and recovery for all it's functionality. It's fully modular, so new functionality can be added.
For the Hackweek, I want to dedicate some time to picotm. I want to finish some of the refactoring work that I have been working on. If there's time left, I'd like to investigate two-phase commits and how to support them in picotm.
Picotm is available at http://picotm.org/.
pudc - A PID 1 process that barks to the internet by mssola
Description
As a fun exercise in order to dig deeper into the Linux kernel, its interfaces, the RISC-V architecture, and all the dragons in between; I'm building a blog site cooked like this:
- The backend is written in a mixture of C and RISC-V assembly.
- The backend is actually PID1 (for real, not within a container).
- We poll and parse incoming HTTP requests ourselves.
- The frontend is a mere HTML page with htmx.
The project is meant to be Linux-specific, so I'm going to use io_uring, pidfs, namespaces, and Linux-specific features in order to drive all of this.
I'm open for suggestions and so on, but this is meant to be a solo project, as this is more of a learning exercise for me than anything else.
Goals
- Have a better understanding of different Linux features from user space down to the kernel internals.
- Most importantly: have fun.
Resources
Add a machine-readable output to dmidecode by jdelvare
Description
There have been repeated requests for a machine-friendly dmidecode output over the last decade. During Hack Week 19, 5 years ago, I prepared the code to support alternative output formats, but didn't have the time to go further. Last year, Jiri Hnidek from Red Hat Linux posted a proof-of-concept implementation to add JSON output support. This is a fairly large pull request which needs to be carefully reviewed and tested.
Goals
Review Jiri's work and provide constructive feedback. Merge the code if acceptable. Evaluate the costs and benefits of using a library such as json-c.
x64id: An x86/x64 instruction disassembler by m.crivellari
Description
This is an old side project. An x86/x64 machine code decoder. It is useful to get instructions' length and identify each of its fields.
Example:
C7 85 68 FF FF FF 00 00 00 00
This is the instruction:
MOV DWORD PTR SS:[LOCAL.38],0
What follows are some of the information collected by the disassembler, based on the specific instruction:
RAW bytes (hex): C7 85 68 FF FF FF 00 00 00 00
Instr. length: 10
Print instruction fields:
Located Prefixes 0:
OP: 0xC7
mod_reg_rm: 0x85
disp (4): 0xFFFFFF68
Iimm: 0x0
Lacks the mnemonic representation: from the previous machine code is not able to produce the "MOV..." instruction, for example.
Goals
The goal is almost easy: partially implement the mnemonic representation. I have already started during the weekend, likely tomorrow I will push the branch!
Resources
- The project: https://github.com/DispatchCode/x64-Instruction-Decoder/
- This is useful to avoid gdb and objdump in local: https://defuse.ca/online-x86-assembler.htm
- Another interesting resource is https://godbolt.org/
Progress
- An initial implementation can be found at: https://github.com/DispatchCode/x64-Instruction-Decoder/tree/mnemonic-support It is described under the "Mnemonic translation" in the README file!
Let's consider this example:
[...other bytes...] 43 89 44 B5 00 01 00 [...other bytes...]
Smart lighting with Pico 2 by jmodak
Description
I am trying to create a smart-lighting project with a Raspberry Pi Pico that reacts to a movie's visuals and audio that involves combining two distinct functions: ambient screen lighting(visual response) and sound-reactive lighting(audio response)
Goals
- Visuals: Capturing the screen's colour requires an external device to analyse screen content and send colour data to the MCU via serial communication.
- Audio: A sound sensor module connected directly to the Pico that can detect sound volume.
- Pico 2W: The MCU receives data fro, both inputs and controls an LED strip.
Resources
- Raspberry Pi Pico 2 W
- RGB LED strip
- Sound detecting sensor
- Power supply
- breadboard and wires
GHC-9.14 and split Hadrian from GHC build by osukup
Description
Prepare openSUSE Tumbleweed project for new GHC Haskell compiler and separate builder (Hadrian) from GHC build
Goals
- have GHC-9.14 project with working compiler and if possible filled with packageset
- have Hadrian in own package built with bootstrap compiler to separate Hadrian bootstrap from ghc bootstrap
Resources
opensuse haskell package gen project
openSUSE on ZoL from OpenZFS project by jkohoutek
Idea is to have SUSE system with OpenZFS as root FS.
Why ZFS
Ways in which ZFS is better than BTRFS
Main goal
Have OpenZFS as install option in the installer and utilize zedenv Boot Environment Manager for SUSE updates install
Goals
- synergy of ZFS with dracut, so snapshots are correctly added to the grub
- synergy of zedenv with zypper
- before every update snapshot is created
- when new kernel or other package which requires reboot is about to be installed, the update will be processed to the new boot environment snapshot and grub configuration changed to boot to this new one
- integrate Root on ZFS as install option to the YaST
- configure Kiwi for the ZFS install images
Completed goals
- prepare ZFS pool compatible with openSUSE installation ✓
- install openSUSE with root on ZFS ✓
- boot to the prepared and installed system ✓
Resources:
RMT.rs: High-Performance Registration Path for RMT using Rust by gbasso
Description
The SUSE Repository Mirroring Tool (RMT) is a critical component for managing software updates and subscriptions, especially for our Public Cloud Team (PCT). In a cloud environment, hundreds or even thousands of new SUSE instances (VPS/EC2) can be provisioned simultaneously. Each new instance attempts to register against an RMT server, creating a "thundering herd" scenario.
We have observed that the current RMT server, written in Ruby, faces performance issues under this high-concurrency registration load. This can lead to request overhead, slow registration times, and outright registration failures, delaying the readiness of new cloud instances.
This Hackweek project aims to explore a solution by re-implementing the performance-critical registration path in Rust. The goal is to leverage Rust's high performance, memory safety, and first-class concurrency handling to create an alternative registration endpoint that is fast, reliable, and can gracefully manage massive, simultaneous request spikes.
The new Rust module will be integrated into the existing RMT Ruby application, allowing us to directly compare the performance of both implementations.
Goals
The primary objective is to build and benchmark a high-performance Rust-based alternative for the RMT server registration endpoint.
Key goals for the week:
- Analyze & Identify: Dive into the
SUSE/rmtRuby codebase to identify and map out the exact critical path for server registration (e.g., controllers, services, database interactions). - Develop in Rust: Implement a functionally equivalent version of this registration logic in Rust.
- Integrate: Explore and implement a method for Ruby/Rust integration to "hot-wire" the new Rust module into the RMT application. This may involve using FFI, or libraries like
rb-sysormagnus. - Benchmark: Create a benchmarking script (e.g., using
k6,ab, or a custom tool) that simulates the high-concurrency registration load from thousands of clients. - Compare & Present: Conduct a comparative performance analysis (requests per second, latency, success/error rates, CPU/memory usage) between the original Ruby path and the new Rust path. The deliverable will be this data and a summary of the findings.
Resources
- RMT Source Code (Ruby):
https://github.com/SUSE/rmt
- RMT Documentation:
https://documentation.suse.com/sles/15-SP7/html/SLES-all/book-rmt.html
- Tooling & Stacks:
- RMT/Ruby development environment (for running the base RMT)
- Rust development environment (
rustup,cargo)
- Potential Integration Libraries:
- rb-sys:
https://github.com/oxidize-rb/rb-sys - Magnus:
https://github.com/matsadler/magnus
- rb-sys:
- Benchmarking Tools:
k6(https://k6.io/)ab(ApacheBench)
Recipes catalog and calculator in Rails 8 by gfilippetti
My wife needs a website to catalog and sell the products of her upcoming bakery, and I need to learn and practice modern Rails. So I'm using this Hack Week to build a modern store using the latest Ruby on Rails best practices, ideally up to the deployment.
TO DO
- Index page
- Product page
- Admin area -- Supplies calculator based on orders -- Orders notification
- Authentication
- Payment
- Deployment
Day 1
As my Rails knowledge was pretty outdated and I had 0 experience with Turbo (wich I want to use in the app), I started following a turbo-rails course. I completed 5 of 11 chapters.
Day 2
Continued the course until chapter 8 and added live updates & an empty state to the app. I should finish the course on day 3 and start my own project with the knowledge from it.
Hackweek 25
For this Hackweek I'll continue this project, focusing on a Catalog/Calculator for my wife's recipes so she can use for her Café.
Day 1