Project Description
This project aims to build a unikernel that has a sole purpose: run WebAssembly applications.
Unikernels offer a limited attack surface and theoretically can provide better performance compared to running the same application on top of a regular operating system. However, running applications on top of unikernels is not an easy tasks. Some might just work out of the box, others will require quite some efforts to be ported.
WebAssembly is a binary application format. A developer write their application using their favorite programming language, then compile it using the WebAssembly target. This produces a Wasm module, that can be run on top of a WebAssembly runtime. The runtime abstracts the underlying operating system and architecture, making the Wasm module capable of running everywhere. The WebAssembly runtime is also in charge of exposing a series of capabilities that can be leveraged by the WebAssembly guest. These capabilities could be system-related (like interacting with the filesystem, this is what WASI does) or could be functionalities provided by a specific platform (like accessing Key/Value stores, implementing http endpoints,... this is what platforms like Spin or Spiderlightning offer).
Why should we have a unikernel that runs WebAssembly:
- To improve security. Each WebAssembly runtime executes the WebAssembly code inside of a dedicated sandbox. This isolates the Wasm workloads from each other and from the host. However this is all done in software. By using a unikernel, we will be running the application on a host (the unikernel) that as a limited attack surface, plus it will be placed inside of a traditional VM that offers better isolation capabilities.
- To go fast. We could just place a regular WebAssembly runtime inside of a VM and call it a day (like running that inside of kata for example). However, having a super limited use case, I think we can build a really optimized system (à la firecraker)
Why I think this has higher chances of adoption compared to traditional "wanna run everything" unikernels:
- WebAssembly modules have clear boundaries. The set of capabilities offered is set by the platform (Spin, Spiderlightning) is limited. That reduces the amount of scenarios the unikernel must support.
- A better developer experience. A developer will build an application targeting a certain platform (Spin, Spiderlightning). This will produce a Wasm module that can be run, unchanged, by a regular WebAssembly runtime, including this unikernel. Both the regular WebAssembly runtime and the unikernel will implement the same set of capabilities. The WebAssembly guest won't even notice where its being ran.
Why am I doing this
I've always been fascinated by unikernels and I'm deeply into WebAssembly. I think there's a purpose for the two things to play together, as described above.
State of the project
I've a running POC. The unikernel is based on rusty-hermit, a unikernel written in Rust.
I've a unikernel application that is capable of running the unmodified WebAssembly module produced by the Spiderlightning key-value demo.
The unikernel is running inside of qemu, using kvm.
Goal for this Hackweek
Main goal - run WebAssembly
The main goal is be able to run the http server demo from Spiderlightning. I don't care about being able to run WASI modules, right now I just want to target some of the interfaces offered by Spiderlightning.
I don't need external help to achieve this goal.
Virtualization goals - HELP NEEDED
As an additional goal, I would like to replace the usage of qemu with something that is custom built.
We need a stripped down hypervisor that:
- Offers a network stack based on virtio - this is what hermit supports
- [Optional] offers a virtio disk backed by virtiofs - that's the only type of disk supported by hermit unikernel
We could leverage the Rust VMM libraries (the ones used by firecraker and crosvm) to build it, use cloud-hypervisor or other things you have in mind. It would be great to have all of that written in Rust, because it would allow me to reuse other Wasm/OCI-related things I'm already using inside of other projects I maintain/contribute to.
First step
Have a cli program that works in this way:
console
run
This would setup the network stack and run the unikernel image provided by the cli.
In this case we don't care about the unikernel being run. It could be the "hello world" example from hermit or it could be the wasm unikernel I'm building. In the latter case, the unikernel image would also bundle inside of itself the actual WebAssembly module.
Second step
Have a cli program that works in this way:
console
run
The binary will now bundle the wasm unikernel application, hence it's no longer required to specify that on the cli. Instead, on the cli the user will point us to the WebAssembly module to run.
The module will somehow be injected into the VM and consumed by the unikernel. How to do that, I have some ideas, but you're the virtualization expert, I rely on you to help me with that 
.
This makes the unikernel image generic, because there's no need to bundle the actual WebAssembly module to execute inside of it.
Resources
I've still to upload the POC on GitHub. In the meantime these are the Virtualization related links:
- Intro to hermit unikernel
- A hypervisor built by the hermit people: this is not meant to be used in production
- hermit wiki: it has useful resources, especially this one
- Bootloader to be used with qemu
- VMM reference implementation
- cloud-hypervisor
Looking for hackers with the skills:
This project is part of:
Hack Week 22
Activity
Comments
-
almost 2 years ago by flavio_castelli | Reply
Project update
I started writing a series of blog post about this project. You can find the first one here.
I got a POC working, I managed to achieve the "main" goal, but I didn't have time/help to accomplish the secondary virtualization goals.
You can find the working prototype here
Similar Projects
SUSE KVM Best Practices by roseswe
Description
SUSE Best Practices around KVM, especially for SAP workloads. Early Google presentation already made from various customer projects and SUSE sources.
Goals
Complete presentation we can reuse in SUSE Consulting projects
Resources
KVM (virt-manager) images
SUSE/SAP/KVM Best Practices
- https://documentation.suse.com/en-us/sles/15-SP6/single-html/SLES-virtualization/
- SAP Note 1522993 - "Linux: SAP on SUSE KVM - Kernel-based Virtual Machine" && 2284516 - SAP HANA virtualized on SUSE Linux Enterprise hypervisors https://me.sap.com/notes/2284516
- SUSECon24: [TUTORIAL-1253] Virtualizing SAP workloads with SUSE KVM || https://youtu.be/PTkpRVpX2PM
- SUSE Best Practices for SAP HANA on KVM - https://documentation.suse.com/sbp/sap-15/html/SBP-SLES4SAP-HANAonKVM-SLES15SP4/index.html
Extending KubeVirtBMC's capability by adding Redfish support by zchang
Description
In Hack Week 23, we delivered a project called KubeBMC (renamed to KubeVirtBMC now), which brings the good old-fashioned IPMI ways to manage virtual machines running on KubeVirt-powered clusters. This opens the possibility of integrating existing bare-metal provisioning solutions like Tinkerbell with virtualized environments. We even received an inquiry about transferring the project to the KubeVirt organization. So, a proposal was filed, which was accepted by the KubeVirt community, and the project was renamed after that. We have many tasks on our to-do list. Some of them are administrative tasks; some are feature-related. One of the most requested features is Redfish support.
Goals
Extend the capability of KubeVirtBMC by adding Redfish support. Currently, the virtbmc component only exposes IPMI endpoints. We need to implement another simulator to expose Redfish endpoints, as we did with the IPMI module. We aim at a basic set of functionalities:
- Power management
- Boot device selection
- Virtual media mount (this one is not so basic )
Resources
Harvester Packer Plugin by mrohrich
Description
Hashicorp Packer is an automation tool that allows automatic customized VM image builds - assuming the user has a virtualization tool at their disposal. To make use of Harvester as such a virtualization tool a plugin for Packer needs to be written. With this plugin users could make use of their Harvester cluster to build customized VM images, something they likely want to do if they have a Harvester cluster.
Goals
Write a Packer plugin bridging the gap between Harvester and Packer. Users should be able to create customized VM images using Packer and Harvester with no need to utilize another virtualization platform.
Resources
Hashicorp documentation for building custom plugins for Packer https://developer.hashicorp.com/packer/docs/plugins/creation/custom-builders
Source repository of the Harvester Packer plugin https://github.com/m-ildefons/harvester-packer-plugin
Contribute to terraform-provider-libvirt by pinvernizzi
Description
The SUSE Manager (SUMA) teams' main tool for infrastructure automation, Sumaform, largely relies on terraform-provider-libvirt. That provider is also widely used by other teams, both inside and outside SUSE.
It would be good to help the maintainers of this project and give back to the community around it, after all the amazing work that has been already done.
If you're interested in any of infrastructure automation, Terraform, virtualization, tooling development, Go (...) it is also a good chance to learn a bit about them all by putting your hands on an interesting, real-use-case and complex project.
Goals
- Get more familiar with Terraform provider development and libvirt bindings in Go
- Solve some issues and/or implement some features
- Get in touch with the community around the project
Resources
- CONTRIBUTING readme
- Go libvirt library in use by the project
- Terraform plugin development
- "Good first issue" list
Write an url shortener in Rust (And learn in the way) by szarate
So I have 469.icu :), it's currently doing nothing... (and for sale) but in the meantime, I'd like to write an url shortener from scratch and deploy it on my own server
https://github.com/foursixnine/url-manager-rs/tree/main
Agama installer on-line demo by lslezak
Description
The Agama installer provides a quite complex user interface. We have some screenshots on the web page but as it is basically a web application it would be nice to have some on-line demo where users could click and check it live.
The problem is that the Agama server directly accesses the hardware (storage probing) and loads installation repositories. We cannot easily mock this in the on-line demo so the easiest way is to have just a read-only demo. You could explore the configuration options but you could not change anything, all changes would be ignored.
The read-only demo would be a bit limited but I still think it would be useful for potential users get the feeling of the new Agama installer and get familiar with it before using in a real installation.
As a proof of concept I already created this on-line demo.
The implementation basically builds Agama in two modes - recording mode where it saves all REST API responses and replay mode where it for the REST API requests returns the previously recorded responses. Recording in the browser is inconvenient and error prone, there should be some scripting instead (see below).
Goals
- Create an Agama on-line demo which can be easily tested by users
- The Agama installer is still in alpha phase and in active development, the online demo needs to be easily rebuilt with the latest Agama version
- Ideally there should be some automation so the demo page is rebuilt automatically without any developer interactions (once a day or week?)
TODO
- Use OpenAPI to get all Agama REST API endpoints, write a script which queries all the endpoints automatically and saves the collected data to a file (see this related PR).
- Write a script for starting an Agama VM (use libvirt/qemu?), the script should ensure we always use the same virtual HW so if we need to dump the latest REST API state we get the same (or very similar data). This should ensure the demo page does not change much regarding the storage proposal etc...
- Fix changing the product, currently it gets stuck after clicking the "Select" button.
- Move the mocking data (the recorded REST API responses) outside the Agama sources, it's too big and will be probably often updated. To avoid messing the history keep it in a separate GitHub repository
- Allow changing the UI language
- Display some note (watermark) in the page so it is clear it is a read-only demo (probably with some version or build date to know how old it is)
- Automation for building new demo page from the latest sources. There should be some check which ensures the recorded data still matches the OpenAPI specification.
Changing the UI language
This will be quite tricky because selecting the proper translation file is done on the server side. We would probably need to completely re-implement the logic in the browser side and adapt the server for that.
Also some REST API responses contain translated texts (storage proposal, pattern names in software). We would need to query the respective endpoints in all supported languages and return the correct response in runtime according to the currently selected language.
Resources
- Agama sources
- Experimental proof of concept demo
- The respective source code change
Better diff'ing experience by MSirringhaus
Description
For diff-ing directories, I usually like to use meld, but it struggles a lot with large trees. Experiment with writing a TUI meld-clone for diffing directories and files
Goals
Get first prototype going of a TUI that can show
- diffs of text-files
- diffs of directories.
Stretch goals
- Themes
- Filters (no whitespace, etc.)
- Live config changes (Show/hide line numbers, etc.)
Grapesss: a physical Shamir's Secret Sharing application [ESP32-C3 + Mobile] by ecandino
Description
A couple of years ago I created StegoSecretS, a small cli used to encrypt and split a secret into multiple keys, using the Shamir's Secret Sharing algorithm.
The idea is to re-implement the project using physical devices. One device alone will be useless, but when close together they can be used to decrypt the secret.
On a practical side the user encrypts the secret with a mobile application. The same application is used to split the secret, and load the partial keys into different micro-controllers. Another user will be able to decrypt the secret only having at least N devices close together (using the application).
I'm planning to use a couple of ESP32-C3 I bought, and build a very simple Android mobile application.
Goals
- Learn about Rust and micro-controllers (ESP32-C3)
- Learn about mobile applications (Android and Kotlin)
Resources
Hack on isotest-ng - a rust port of isotovideo (os-autoinst aka testrunner of openQA) by szarate
Description
Some time ago, I managed to convince ByteOtter to hack something that resembles isotovideo but in Rust, not because I believe that Perl is dead, but more because there are certain limitations in the perl code (how it was written), and its always hard to add new functionalities when they are about implementing a new backend, or fixing bugs (Along with people complaining that Perl is dead, and that they don't like it)
In reality, I wanted to see if this could be done, and ByteOtter proved that it could be, while doing an amazing job at hacking a vnc console, and helping me understand better what RuPerl needs to work.
I plan to keep working on this for the next few years, and while I don't aim for feature completion or replacing isotovideo tih isotest-ng (name in progress), I do plan to be able to use it on a daily basis, using specialized tooling with interfaces, instead of reimplementing everything in the backend
Todo
- Add
maketargets for testability, e.g "spawn qemu and type" - Add image search matching algorithm
- Add a Null test distribution provider
- Add a Perl Test Distribution Provider
- Fix unittests https://github.com/os-autoinst/isotest-ng/issues/5
- Research OpenTofu how to add new hypervisors/baremetal to OpenTofu
- Add an interface to openQA cli
Goals
- Implement at least one of the above, prepare proposals for GSoC
- Boot a system via it's BMC
Resources
See https://github.com/os-autoinst/isotest-ng