knots
a project by LarsMB
Updated over 3 years ago. 2 hacker ♥️. 2 followers.

Summary

knots aims to be a multi-node S3 benchmark and load generator written in Rust and on top of WebAssembly, waPC, and waSCC.

Goal for this Hackweek

My goal for HackWeek is to have a prototype actor that can at least run a few GET/PUT requests against a single bucket and tells me how long that took.

Collaboration

Since this is a training wheels exercise for me on multiple fronts (Rust, and the entire WASM/waPC/waSCC world), I'm not actively looking for other contributors yet. I don't learn as much if someone else does the work, and I have precisely zero experience with any of the technologies involved except S3 ;-)

But if you have experience, I don't mind chatting!

Especially if your project maybe includes a krustlet/waSCC ops environment to try and deploy the actor on.

I just want an S3 benchmark, now

I recommend you take a look at these:

  • fio is an excellent C tool for benchmarking all sorts of data IO mechanisms, and is very useful for comparing protocols. I contributed a simple S3 module to it a while ago. If your primary interest is data throughput and latency, fio is your tool of choice.
  • MinIO warp is a versatile tool written in Go for benchmarking various aspects of S3 and probably the most mature choice.
  • cosbench is a very well-known tool for benchmarking S3 written in Java.

Join this project

Looking for hackers with the skills:

s3 wasm wascc wasi ceph krustlet rust containerless

This project is part of:

Hack Week 20

Activity

  • almost 4 years ago: jluis liked this project.
  • almost 4 years ago: LarsMB liked this project.
  • almost 4 years ago: LarsMB added keyword "containerless" to this project.
  • almost 4 years ago: LarsMB started this project.
  • almost 4 years ago: LarsMB added keyword "ceph" to this project.
  • almost 4 years ago: LarsMB added keyword "krustlet" to this project.
  • almost 4 years ago: LarsMB added keyword "rust" to this project.
  • almost 4 years ago: LarsMB added keyword "s3" to this project.
  • almost 4 years ago: LarsMB added keyword "wasm" to this project.
  • almost 4 years ago: LarsMB added keyword "wascc" to this project.
  • almost 4 years ago: LarsMB added keyword "wasi" to this project.
  • almost 4 years ago: LarsMB originated this project.


    • Comments

    Be the first to comment!

    Similar Projects

    Hack on isotest-ng - a rust port of isotovideo (os-autoinst aka testrunner of openQA) by szarate

    Description

    Some time ago, I managed to convince ByteOtter to hack something that resembles isotovideo but in Rust, not because I believe that Perl is dead, but more because there are certain limitations in the perl code (how it was written), and its always hard to add new functionalities when they are about implementing a new backend, or fixing bugs (Along with people complaining that Perl is dead, and that they don't like it)

    In reality, I wanted to see if this could be done, and ByteOtter proved that it could be, while doing an amazing job at hacking a vnc console, and helping me understand better what RuPerl needs to work.

    I plan to keep working on this for the next few years, and while I don't aim for feature completion or replacing isotovideo tih isotest-ng (name in progress), I do plan to be able to use it on a daily basis, using specialized tooling with interfaces, instead of reimplementing everything in the backend

    Todo

    • Add make targets for testability, e.g "spawn qemu and type"
    • Add image search matching algorithm
    • Add a Null test distribution provider
    • Add a Perl Test Distribution Provider
    • Fix unittests https://github.com/os-autoinst/isotest-ng/issues/5
    • Research OpenTofu how to add new hypervisors/baremetal to OpenTofu
    • Add an interface to openQA cli

    Goals

    • Implement at least one of the above, prepare proposals for GSoC
    • Boot a system via it's BMC

    Resources

    See https://github.com/os-autoinst/isotest-ng

    SMB3 Server written entirely in Rust by dmulder

    Description

    Given the number of bugs frequently discovered in the Samba code caused by memory issues, it makes sense to re-write the smbd service purely in Rust code. Meanwhile, it would be wise to abandon backwards compatibility here with insecure protocol versions, and simply implement the SMB3 spec.

    Goals

    Get a simple server up and running and get it merged into upstream Samba (which now has Rust build support).

    Resources

    Kanidm: A safe and modern IDM system by firstyear

    Kanidm is an IDM system written in Rust for modern systems authentication. The github repo has a detailed "getting started" on the readme.

    Kanidm Github

    In addition Kanidm has spawn a number of adjacent projects in the Rust ecosystem such as LDAP, Kerberos, Webauthn, and cryptography libraries.

    In this hack week, we'll be working on Quokca, a certificate authority that supports PKCS11/TPM storage of keys, issuance of PIV certificates, and ACME without the feature gatekeeping implemented by other CA's like smallstep.

    For anyone who wants to participate in Kanidm, we have documentation and developer guides which can help.

    I'm happy to help and share more, so please get in touch!

    Implement a CLI tool for Trento - trentoctl by nkopliku

    Description

    Implement a trentoctl CLI for interacting with a trento installation

    Goals

    • learn rust
    • implement an initial trentoctl tool to enhance trento automation
    • have fun

    Resources

    trento rust. TUIs listed on this other hackweek project Hack on rich terminal user interfaces

    Write an url shortener in Rust (And learn in the way) by szarate

    So I have 469.icu :), it's currently doing nothing... (and for sale) but in the meantime, I'd like to write an url shortener from scratch and deploy it on my own server

    https://github.com/foursixnine/url-manager-rs/tree/main