cosmic-river
a project by dmaiocchi
Updated almost 6 years ago. 1 hackers ♥️. 1 follower.

Rationale:

cosmic-river aims distributed Event-Driven workflows as generic, composable tool

motivation

I want to gain more pragmatic knowledge on following for me unfamiliar concepts:

  • Message Brokers: https://en.wikipedia.org/wiki/Message_broker

  • I will build the project with Clojure, a Lisp dialect with code-as-data philosophy. (https://clojure.org/)

The project itself has already a design and a scope but I will update it step by step for allowing freedom to experiment.

https://en.wikipedia.org/wiki/Cosmic_ocean

Join this project

Looking for hackers with the skills:

clojure jvm functionalprogramming lisp distributedsystems distribution clojurescript

This project is part of:

Hack Week 18

Activity

  • almost 6 years ago: dmaiocchi liked this project.
  • almost 6 years ago: dmaiocchi added keyword "clojurescript" to this project.
  • almost 6 years ago: dmaiocchi added keyword "clojure" to this project.
  • almost 6 years ago: dmaiocchi added keyword "jvm" to this project.
  • almost 6 years ago: dmaiocchi added keyword "functionalprogramming" to this project.
  • almost 6 years ago: dmaiocchi added keyword "lisp" to this project.
  • almost 6 years ago: dmaiocchi added keyword "distributedsystems" to this project.
  • almost 6 years ago: dmaiocchi added keyword "distribution" to this project.
  • almost 6 years ago: dmaiocchi started this project.
  • almost 6 years ago: dmaiocchi originated this project.


    • Comments

    • dmaiocchi
      almost 6 years ago by dmaiocchi | Reply

      day 01:

      During the 1st day of my hackweek I achieved the core design and schema of cosmic-river. I have draw the most components and their responsibilities, in a lightweight way.

      It has already some functionality which however cannot be used already.

      In the next days I plan to spend time on the message-brokers parts and this is the unfamiliar part of my hackweek, so it can take some times but still an exciting part ;)

      Thanks to interactive programming I was able to debug, code in a revolutionary workflow. I was using emacs + CIDER.

      https://docs.cider.mx/cider/usage/interactive_programming.html

    • dmaiocchi
      almost 6 years ago by dmaiocchi | Reply

      day 2

      During second day I have learned concept with rabbitmq and message broker. ( I had already some theory background).

      I will focus with rabbitmq but I choosed to make cosmic-river composable, so an user could choose Kafka instead of rabbitmq.

      I have also designed the first skeleton with rabbitmq and continue on this model producer/consumer (server/client).

      In the project itself I needed to refactor things and create two different applicaitication. THe scaffold is ready and i plan to continue to work with.

    • dmaiocchi
      almost 6 years ago by dmaiocchi | Reply

      day 3

      All client/server rabbitmq part was improved.

    • dmaiocchi
      almost 6 years ago by dmaiocchi | Reply

      day4:

      Today I will work on serialization part and implement filters

    • dmaiocchi
      almost 6 years ago by dmaiocchi | Reply

      day 4:

      • Improved documentation and presentation ( allthough I need to document the specification to configuration).
      • Data serialization over network done

      • Refactoring of the various namespaces.

      • Continuing working on the filter for events to be generic and parameterizable

    Similar Projects

    concierge: a distributed object capabilities based secret management tool implemented with Spritely Goblins by gleidi

    Description

    The idea is to build a FOSS command line (for now) secrets management tool, you may have heard of SOPS or Hashicorp Vault, with Spritely Goblins.

    Goblins is a distributed object programming environment. It provides an intuitive security model, automatic local transactions for locally synchronous operations, and an easy to use and efficient asynchronous programming interface for encapsulated objects which can live anywhere on the network. Its networking model abstracts away these details so the programmer can focus on object programming rather than protocol architecture.

    Goblins also integrates powerful distributed debugging tools, and a process persistence and upgrade model which respects its security fundamentals.

    Goals

    • learn more about distributed object capabilities environments
    • create a Goblins application (or more realistically a demo) able to serialize encrypted secrets and allow only the read capability owner to read the serialized secrets.

    Resources

    • https://codeberg.org/fishinthecalculator/concierge
    • https://spritely.institute/goblins/
    • https://www.gnu.org/software/guile/manual/guile.html

    Explore simple and distro indipendent declarative Linux starting on Tumbleweed or Arch Linux by janvhs

    Description

    Inspired by mkosi the idea is to experiment with a declarative approach of defining Linux systems. A lot of tools already make it possible to manage the systems infrastructure by using description files, rather than manual invocation. An example for this are systemd presets for managing enabled services or the /etc/fstab file for describing how partitions should be mounted.

    If we would take inspiration from openSUSE MicroOS and their handling of the /etc/ directory, we could theoretically use systemd-sysupdate to swap out the /usr/ partition and create an A/B boot scheme, where the /usr/ partition is always freshly built according to a central system description. In the best case it would be possible to still utilise snapshots, but an A/B root scheme would be sufficient for the beginning. This way you could get the benefit of NixOS's declarative system definition, but still use the distros package repositories and don't have to deal with the overhead of Flakes or the Nix language.

    Goals

    • A simple and understandable system
    • Check fitness of mkosi or write a simple extensible image builder tool for it
    • Create a declarative system specification
    • Create a system with swappable /usr/ partition
    • Create an A/B root scheme
    • Swap to the new system without reboot (kexec?)

    Resources

    • Ideas that have been floating around in my head for a while
    • https://0pointer.net/blog/fitting-everything-together.html
    • GNOME OS
    • MicroOS
    • systemd mkosi
    • Vanilla OS

    Switch software-o-o to parse repomd data by hennevogel

    Currently software.opensuse.org search is using the OBS binary search for everything, even for packages inside the openSUSE distributions. Let's switch this to use repomd data from download.opensuse.org

    Testing and adding GNU/Linux distributions on Uyuni by juliogonzalezgil

    Join the Gitter channel! https://gitter.im/uyuni-project/hackweek

    Uyuni is a configuration and infrastructure management tool that saves you time and headaches when you have to manage and update tens, hundreds or even thousands of machines. It also manages configuration, can run audits, build image containers, monitor and much more!

    Currently there are a few distributions that are completely untested on Uyuni or SUSE Manager (AFAIK) or just not tested since a long time, and could be interesting knowing how hard would be working with them and, if possible, fix whatever is broken.

    For newcomers, the easiest distributions are those based on DEB or RPM packages. Distributions with other package formats are doable, but will require adapting the Python and Java code to be able to sync and analyze such packages (and if salt does not support those packages, it will need changes as well). So if you want a distribution with other packages, make sure you are comfortable handling such changes.

    No developer experience? No worries! We had non-developers contributors in the past, and we are ready to help as long as you are willing to learn. If you don't want to code at all, you can also help us preparing the documentation after someone else has the initial code ready, or you could also help with testing :-)

    The idea is testing Salt and Salt-ssh clients, but NOT traditional clients, which are deprecated.

    To consider that a distribution has basic support, we should cover at least (points 3-6 are to be tested for both salt minions and salt ssh minions):

    1. Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
    2. Onboarding (salt minion from UI, salt minion from bootstrap scritp, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator)
    3. Package management (install, remove, update...)
    4. Patching
    5. Applying any basic salt state (including a formula)
    6. Salt remote commands
    7. Bonus point: Java part for product identification, and monitoring enablement
    8. Bonus point: sumaform enablement (https://github.com/uyuni-project/sumaform)
    9. Bonus point: Documentation (https://github.com/uyuni-project/uyuni-docs)
    10. Bonus point: testsuite enablement (https://github.com/uyuni-project/uyuni/tree/master/testsuite)

    If something is breaking: we can try to fix it, but the main idea is research how supported it is right now. Beyond that it's up to each project member how much to hack :-)

    • If you don't have knowledge about some of the steps: ask the team
    • If you still don't know what to do: switch to another distribution and keep testing.

    This card is for EVERYONE, not just developers. Seriously! We had people from other teams helping that were not developers, and added support for Debian and new SUSE Linux Enterprise and openSUSE Leap versions :-)

    Pending

    FUSS

    FUSS is a complete GNU/Linux solution (server, client and desktop/standalone) based on Debian for managing an educational network.

    https://fuss.bz.it/

    Seems to be a Debian 12 derivative, so adding it could be quite easy.

    • [W] Reposync (this will require using spacewalk-common-channels and adding channels to the .ini file)
    • [W] Onboarding (salt minion from UI, salt minion from bootstrap script, and salt-ssh minion) (this will probably require adding OS to the bootstrap repository creator) --> Working for all 3 options (salt minion UI, salt minion bootstrap script and salt-ssh minion from the UI).
    • [W] Package management (install, remove, update...) --> Installing a new package works, needs to test the rest.
    • [I] Patching (if patch information is available, could require writing some code to parse it, but IIRC we have support for Ubuntu already). No patches detected. Do we support patches for Debian at all?
    • [W] Applying any basic salt state (including a formula)
    • [W] Salt remote commands
    • [ ] Bonus point: Java part for product identification, and monitoring enablement