knots
a project by LarsMB
Updated over 4 years ago. 2 hacker ♥️. 2 followers.

Summary

knots aims to be a multi-node S3 benchmark and load generator written in Rust and on top of WebAssembly, waPC, and waSCC.

Goal for this Hackweek

My goal for HackWeek is to have a prototype actor that can at least run a few GET/PUT requests against a single bucket and tells me how long that took.

Collaboration

Since this is a training wheels exercise for me on multiple fronts (Rust, and the entire WASM/waPC/waSCC world), I'm not actively looking for other contributors yet. I don't learn as much if someone else does the work, and I have precisely zero experience with any of the technologies involved except S3 ;-)

But if you have experience, I don't mind chatting!

Especially if your project maybe includes a krustlet/waSCC ops environment to try and deploy the actor on.

I just want an S3 benchmark, now

I recommend you take a look at these:

  • fio is an excellent C tool for benchmarking all sorts of data IO mechanisms, and is very useful for comparing protocols. I contributed a simple S3 module to it a while ago. If your primary interest is data throughput and latency, fio is your tool of choice.
  • MinIO warp is a versatile tool written in Go for benchmarking various aspects of S3 and probably the most mature choice.
  • cosbench is a very well-known tool for benchmarking S3 written in Java.

Join this project

Looking for hackers with the skills:

s3 wasm wascc wasi ceph krustlet rust containerless

This project is part of:

Hack Week 20

Activity

  • over 4 years ago: jluis liked this project.
  • over 4 years ago: LarsMB liked this project.
  • over 4 years ago: LarsMB added keyword "containerless" to this project.
  • over 4 years ago: LarsMB started this project.
  • over 4 years ago: LarsMB added keyword "ceph" to this project.
  • over 4 years ago: LarsMB added keyword "krustlet" to this project.
  • over 4 years ago: LarsMB added keyword "rust" to this project.
  • over 4 years ago: LarsMB added keyword "s3" to this project.
  • over 4 years ago: LarsMB added keyword "wasm" to this project.
  • over 4 years ago: LarsMB added keyword "wascc" to this project.
  • over 4 years ago: LarsMB added keyword "wasi" to this project.
  • over 4 years ago: LarsMB originated this project.


    • Comments

    Be the first to comment!

    Similar Projects

    Updatecli Autodiscovery supporting WASM plugins by olblak

    Description

    Updatecli is a Golang Update policy engine that allow to write Update policies in YAML manifest. Updatecli already has a plugin ecosystem for common update strategies such as automating Dockerfile or Kubernetes manifest from Git repositories.

    This is what we call autodiscovery where Updatecli generate manifest and apply them dynamically based on some context.

    Obviously, the Updatecli project doesn't accept plugins specific to an organization.

    I saw project using different languages such as python, C#, or JS to generate those manifest.

    It would be great to be able to share and reuse those specific plugins

    During the HackWeek, I'll hang on the Updatecli matrix channel

    https://matrix.to/#/#Updatecli_community:gitter.im

    Goals

    Implement autodiscovery plugins using WASM. I am planning to experiment with https://github.com/extism/extism

    To build a simple WASM autodiscovery plugin and run it from Updatecli

    Resources

    • https://github.com/extism/extism
    • https://github.com/updatecli/updatecli
    • https://www.updatecli.io/docs/core/autodiscovery/
    • https://matrix.to/#/#Updatecli_community:gitter.im

    Looking at Rust if it could be an interesting programming language by jsmeix

    Get some basic understanding of Rust security related features from a general point of view.

    This Hack Week project is not to learn Rust to become a Rust programmer. This might happen later but it is not the goal of this Hack Week project.

    The goal of this Hack Week project is to evaluate if Rust could be an interesting programming language.

    An interesting programming language must make it easier to write code that is correct and stays correct when over time others maintain and enhance it than the opposite.

    Learn how to use the Relm4 Rust GUI crate by xiaoguang_wang

    Relm4 is based on gtk4-rs and compatible with libadwaita. The gtk4-rs crate provides all the tools necessary to develop applications. Building on this foundation, Relm4 makes developing more idiomatic, simpler, and faster.

    https://github.com/Relm4/Relm4

    OpenPlatform Self-Service Portal by tmuntan1

    Description

    In SUSE IT, we developed an internal developer platform for our engineers using SUSE technologies such as RKE2, SUSE Virtualization, and Rancher. While it works well for our existing users, the onboarding process could be better.

    To improve our customer experience, I would like to build a self-service portal to make it easy for people to accomplish common actions. To get started, I would have the portal create Jira SD tickets for our customers to have better information in our tickets, but eventually I want to add automation to reduce our workload.

    Goals

    • Build a frontend website (Angular) that helps customers create Jira SD tickets.
    • Build a backend (Rust with Axum) for the backend, which would do all the hard work for the frontend.

    Resources (SUSE VPN only)

    • development site: https://ui-dev.openplatform.suse.com/login?returnUrl=%2Fopenplatform%2Fforms
    • https://gitlab.suse.de/itpe/core/open-platform/op-portal/backend
    • https://gitlab.suse.de/itpe/core/open-platform/op-portal/frontend

    Arcticwolf - A rust based user space NFS server by vcheng

    Description

    Rust has similar performance to C. Also, have a better async IO module and high integration with io_uring. This project aims to develop a user-space NFS server based on Rust.

    Goals

    • Get an understanding of how cargo works
    • Get an understanding of how XDR was generated with xdrgen
    • Create the RUST-based NFS server that supports basic operations like mount/readdir/read/write

    Result (2025 Hackweek)

    • In progress PR: https://github.com/Vicente-Cheng/arcticwolf/pull/1

    Resources

    https://github.com/Vicente-Cheng/arcticwolf

    Build a terminal user-interface (TUI) for Agama by IGonzalezSosa

    Description

    Officially, Agama offers two different user interfaces. On the one hand, we have the web-based interface, which is the one you see when you run the installation media. On the other hand, we have a command-line interface. In both cases, you can use them using a remote system, either using a browser or the agama CLI.

    We would expect most of the cases to be covered by this approach. However, if you cannot use the web-based interface and, for some reason, you cannot access the system through the network, your only option is to use the CLI. This interface offers a mechanism to modify Agama's configuration using an editor (vim, by default), but perhaps you might want to have a more user-friendly way.

    Goals

    The main goal of this project is to built a minimal terminal user-interface for Agama. This interface will allow the user to install the system providing just a few settings (selecting a product, a storage device and a user password). Then it should report the installation progress.

    Resources

    • https://agama-project.github.io/
    • https://ratatui.rs/

    Conclusions

    We have summarized our conclusions in a pull request. It includes screenshots ;-) We did not implement all the features we wanted, but we learn a lot during the process. We know that, if needed, we could write a TUI for Agama and we have an idea about how to build it. Good enough.