Multi-cloud Lightweight metadata CLI
a project by seanmarlow
Updated over 2 years ago. 3 hacker ♥️. 3 followers.

Project Description

The goal of this project is to create a lightweight multi-cloud metadata CLI for Public Cloud environments. There are cloud specific packages that exist but they all have different API and many are developed in Python which is quite heavy for cloud images, especially containers. Leveraging a compiled language will help with keeping the CLI lightweight.

Goal for this Hackweek

  • A multi-cloud metadata package that works in all cloud frameworks (as support is added)
  • Developed in Go to keep the package and dependencies lightweight
  • Developed using Plugin Oriented Programming style to allow for easy inclusion of new Cloud Frameworks. Ideally with a simple config file (no new code).

Resources

  • Go (https://go.dev/)
  • jmespath (https://jmespath.org/)
  • POP (https://pop-book.readthedocs.io/en/latest/)
  • TBD

Join this project

Looking for hackers with the skills:

golang cloud publiccloud cli

This project is part of:

Hack Week 21

Activity

  • over 2 years ago: ucaliskan liked this project.
  • over 2 years ago: amunoz joined this project.
  • over 2 years ago: amunoz liked this project.
  • over 2 years ago: jesus_bv joined this project.
  • over 2 years ago: RicardoFelipeKlein liked this project.
  • almost 3 years ago: seanmarlow started this project.
  • almost 3 years ago: seanmarlow added keyword "golang" to this project.
  • almost 3 years ago: seanmarlow added keyword "cloud" to this project.
  • almost 3 years ago: seanmarlow added keyword "publiccloud" to this project.
  • almost 3 years ago: seanmarlow added keyword "cli" to this project.
  • almost 3 years ago: seanmarlow originated this project.


    • Comments

    • seanmarlow
      over 2 years ago by seanmarlow | Reply

      Unfortunately I will only have a couple days to work on this for Hackweek. Otherwise it would be nice to coordinate the project with anyone interested in joining. What I plan to get to is:

      • Getting started with Golang
      • "Psuedo" programming the basic concept in Python
      • Put together yaml config files for the 3 major CSPs (Azure, AWS, GCP)

      The idea is that there's hopefully no new code to add a csp. The config files would map attr names to a jmespath query such as:

      azure.yaml metadata-url: http://169.254.169.254/metadata/instance?api-version=2021-02-01 private-ipv4-addrs: network.interface[].ipv4.ipAddress[].privateIpAddress private-ipv6-addrs: network.interface[].ipv6.ipAddress[].privateIpAddress private-ip-addrs: network.interface[].*.ipAddress[][].privateIpAddress region: compute.location

    • seanmarlow
      over 2 years ago by seanmarlow | Reply

      I think the challenge will be AWS as Azure and GCE both have ways to get the entire metadata dictionary with a single get request. Afaik AWS doesn't provide this functionality.

    • seanmarlow
      over 2 years ago by seanmarlow | Reply

      It gets a bit complex to query certain items such as the Azure image URN:

      compute.storageProfile.imageReference.[publisher,offer,sku,version] | join(':', @)

      And as above (after ipAddress) when you do a wildcard for some reason it adds an extra nested list to the next object:

      network.interface[].*.ipAddress[][].privateIpAddress

    Similar Projects

    Hack on rich terminal user interfaces by amanzini

    Description

    TUIs (Textual User Interface) are a big classic of our daily workflow. Many linux users 'live' in the terminal and modern implementations have a lot to offer : unicode fonts, 24 bit colors etc.

    Goals

    • Explore the current available solution on modern languages and implement a PoC , for example a small maze generator, porting of a classic game or just display the HackWeek cute logo.
    • Practice some Go / Rust coding and programming patterns
    • Fiddle around, hack, learn, have fun
    • keep a development diary, practice on project documentation

    Follow this link for source code repository

    Some ideas for inspiration:

    Related projects:

    Resources

    Harvester Packer Plugin by mrohrich

    Description

    Hashicorp Packer is an automation tool that allows automatic customized VM image builds - assuming the user has a virtualization tool at their disposal. To make use of Harvester as such a virtualization tool a plugin for Packer needs to be written. With this plugin users could make use of their Harvester cluster to build customized VM images, something they likely want to do if they have a Harvester cluster.

    Goals

    Write a Packer plugin bridging the gap between Harvester and Packer. Users should be able to create customized VM images using Packer and Harvester with no need to utilize another virtualization platform.

    Resources

    Hashicorp documentation for building custom plugins for Packer https://developer.hashicorp.com/packer/docs/plugins/creation/custom-builders

    Source repository of the Harvester Packer plugin https://github.com/m-ildefons/harvester-packer-plugin

    suse-rancher-supportconfig by eminguez

    Description

    Update: Live at https://github.com/e-minguez/suse-rancher-supportconfig I finally didn't used golang but used gum instead add-emoji

    SUSE's supportconfig support tool collects data from the SUSE Operating system. Rancher's rancher2_logs_collector.sh support tool does the same for RKE2/K3s.

    Wouldn't be nice to have a way to run both and collect all data for SUSE based RKE2/K3s clusters? Wouldn't be even better with a fancy TUI tool like bubbletea?

    Ideally the output should be an html page where you can see the logs/data directly from the browser.

    Goals

    • Familiarize myself with both supportconfig and rancher2_logs_collector.sh tools
    • Refresh my golang knowledge
    • Have something that works at the end of the hackweek ("works" may vary add-emoji )
    • Be better in naming things

    Resources

    All links provided above as well as huh

    Automate PR process by idplscalabrini

    Description

    This project is to streamline and enhance the pr review process by adding automation for identifying some issues like missing comments, identifying sensitive information in the PRs like credentials. etc. By leveraging GitHub Actions and golang hooks we can focus more on high-level reviews

    Goals

    • Automate lints and code validations on Github actions
    • Automate code validation on hook
    • Implement a bot to pre-review the PRs

    Resources

    Golang hooks and Github actions

    toptop - a top clone written in Go by dshah

    Description

    toptop is a clone of Linux's top CLI tool, but written in Go.

    Goals

    Learn more about Go (mainly bubbletea) and Linux

    Resources

    GitHub

    Save pytorch models in OCI registries by jguilhermevanz

    Description

    A prerequisite for running applications in a cloud environment is the presence of a container registry. Another common scenario is users performing machine learning workloads in such environments. However, these types of workloads require dedicated infrastructure to run properly. We can leverage these two facts to help users save resources by storing their machine learning models in OCI registries, similar to how we handle some WebAssembly modules. This approach will save users the resources typically required for a machine learning model repository for the applications they need to run.

    Goals

    Allow PyTorch users to save and load machine learning models in OCI registries.

    Resources

    Mortgage Plan Analyzer by RMestre

    https://github.com/rjpmestre/mortgage-plan-analyzer

    Project Description

    Many people face challenges when trying to renegotiate their mortgages with different banks. They receive offers from multiple lenders and struggle to compare them effectively. Each proposal may have slightly different terms and data presentation, making it hard to make informed decisions. Additionally, understanding the impact of various taxes and variables can be complex. The Mortgage Plan Analyzer project aims to address these issues.

    Project Overview:

    The Mortgage Plan Analyzer is a web-based tool built using PHP, Laravel, Livewire, and AdminLTE/bootstrap. It provides a user-friendly platform for individuals to input basic specifications about their mortgage, adjust taxes and variables, and obtain short-term projections for each proposal. Users can also compare multiple mortgage offers side by side, enabling them to make informed decisions about their mortgage renegotiation.

    Why Start This Project:

    I found myself in this position and most tools I found around are either for marketing/selling purposes or not flexible enough. As i was starting getting lost in a jungle of spreadsheets i thought I could just create a tool to help me and others that may be experiencing the same struggles to provide clarity and transparency in the decision-making process.

    Hackweek 25 ideas (to refine still :) )

    • Euribor Trends in Projections
    • - Use historical Euribor data to model optimistic and pessimistic scenarios for variable-rate loans.
    • Use the annual summaries (installments, amortizations, etc) and run some analysis to highlight key differences, like short-term savings vs. long-term costs
    • Financial plan can be hard/boring to follow. Create a simple viewing mode that summarizes monthly values and their annual sums.

    Hackweek 24 update

    • Improved summaries graphs by adding:
    • - Line graph;
    • - Accumulated line graph;
    • - Set the range to short/mid/long term;
    • - Highlight best simulation and value per year;
    • Improve the general behaviour of the forms:
    • - Simulations name setting;
    • - Cloning simulations;
    • - Adjust update timing on input changes;
    • Show/Hide big tables;
    • Support multi languages (added english);
    • Added examples;
    • Adjustments to fonts and sizes;
    • Fixed loading screen;
    • Dependencies adjustments;

    Hackweek 23 initial release

    • Developed a base site that:
    • - Allows adding up to 3 simulations;
    • - Create financial plans;
    • - Simulations comparison graph for the first 4 years;
    • Created Github project @ https://github.com/rjpmestre/mortgage-plan-analyzer ;
    • Launched a demo instance using Oracle Cloud Free Tier currently @ http://138.3.251.182/

    Resources

    • Banco de Portugal: Main simulator all portuguese banks have to follow ( https://clientebancario.bportugal.pt/credito-habitacao )
    • Laravel: A PHP web application framework for building robust and scalable applications. ( https://laravel.com/ )
    • Livewire: A Laravel library for building dynamic interfaces without writing JavaScript. ( https://livewire.laravel.com/ )
    • AdminLTE: A responsive admin dashboard template for creating a visually appealing interface. ( https://adminlte.io/ )

    Implement a CLI tool for Trento - trentoctl by nkopliku

    Description

    Implement a trentoctl CLI for interacting with a trento installation

    Goals

    • learn rust
    • implement an initial trentoctl tool to enhance trento automation
    • have fun

    Resources

    trento rust. TUIs listed on this other hackweek project Hack on rich terminal user interfaces

    suse-rancher-supportconfig by eminguez

    Description

    Update: Live at https://github.com/e-minguez/suse-rancher-supportconfig I finally didn't used golang but used gum instead add-emoji

    SUSE's supportconfig support tool collects data from the SUSE Operating system. Rancher's rancher2_logs_collector.sh support tool does the same for RKE2/K3s.

    Wouldn't be nice to have a way to run both and collect all data for SUSE based RKE2/K3s clusters? Wouldn't be even better with a fancy TUI tool like bubbletea?

    Ideally the output should be an html page where you can see the logs/data directly from the browser.

    Goals

    • Familiarize myself with both supportconfig and rancher2_logs_collector.sh tools
    • Refresh my golang knowledge
    • Have something that works at the end of the hackweek ("works" may vary add-emoji )
    • Be better in naming things

    Resources

    All links provided above as well as huh

    Jenny Static Site Generator by adam.pickering

    Description

    For my personal site I have been using hugo. It works, but I am not satisfied: every time I want to make a change (which is infrequently) I have to read through the documentation again to understand how hugo works. I don't find the documentation easy to use, and the structure of the repository that hugo requires is unintuitive/more complex than what I need. So, I have decided to write my own simple static site generator in Go. It is named Jenny, after my wife.

    Goals

    • Pages can be written in markdown (which is automatically converted to HTML), but other file types are also allowed
    • Easy to understand and use
      • Intuitive, simple design
      • Clear documentation
      • Hot reloading
      • Binaries provided for download
    • Future maintenance is easy
      • Automated releases

    Resources

    https://github.com/adamkpickering/jenny

    Dartboard TUI by IValentin

    Description

    Our scalability and performance testing swiss-army knife tool Dartboard is a major WIP so why not add more scope creep? Dartboard is a cli tool which enables users to:

    • Define a "Dart" config file as YAML which defines the various components to be created/setup when Dartboard runs its commands
    • Spin up infrastructure utilizing opentofu/terraform providers
    • Setup K3s or RKE2 clusters on the newly created infrastructure
    • Deploy Rancher (with or without downstream cluster), rancher-monitoring (Grafana + Prometheus)
    • Create resources in-bulk within the newly created Rancher cluster (ConfigMaps, Secrets, Users, Roles, etc.)
    • Run various performance and scalability tests via k6
    • Export/Import various tracked metrics (WIP)

    Given all these features (and the features to come), it can be difficult to onboard and transfer knowledge of the tool. With a TUI, Dartboard's usage complexity can be greatly reduced!

    Goals

    • Create a TUI for Dartboard's "subcommands"
    • Gain more familiarity with Dartboard and create a more user-friendly interface to enable others to use it
    • Stretch Create a TUI workflow for generating a Dart file

    Resources

    https://github.com/charmbracelet/bubbletea

    A CLI for Harvester by mohamed.belgaied

    [comment]: # Harvester does not officially come with a CLI tool, the user is supposed to interact with Harvester mostly through the UI [comment]: # Though it is theoretically possible to use kubectl to interact with Harvester, the manipulation of Kubevirt YAML objects is absolutely not user friendly. [comment]: # Inspired by tools like multipass from Canonical to easily and rapidly create one of multiple VMs, I began the development of Harvester CLI. Currently, it works but Harvester CLI needs some love to be up-to-date with Harvester v1.0.2 and needs some bug fixes and improvements as well.

    Project Description

    Harvester CLI is a command line interface tool written in Go, designed to simplify interfacing with a Harvester cluster as a user. It is especially useful for testing purposes as you can easily and rapidly create VMs in Harvester by providing a simple command such as: harvester vm create my-vm --count 5 to create 5 VMs named my-vm-01 to my-vm-05.

    asciicast

    Harvester CLI is functional but needs a number of improvements: up-to-date functionality with Harvester v1.0.2 (some minor issues right now), modifying the default behaviour to create an opensuse VM instead of an ubuntu VM, solve some bugs, etc.

    Github Repo for Harvester CLI: https://github.com/belgaied2/harvester-cli

    Done in previous Hackweeks

    • Create a Github actions pipeline to automatically integrate Harvester CLI to Homebrew repositories: DONE
    • Automatically package Harvester CLI for OpenSUSE / Redhat RPMs or DEBs: DONE

    Goal for this Hackweek

    The goal for this Hackweek is to bring Harvester CLI up-to-speed with latest Harvester versions (v1.3.X and v1.4.X), and improve the code quality as well as implement some simple features and bug fixes.

    Some nice additions might be: * Improve handling of namespaced objects * Add features, such as network management or Load Balancer creation ? * Add more unit tests and, why not, e2e tests * Improve CI * Improve the overall code quality * Test the program and create issues for it

    Issue list is here: https://github.com/belgaied2/harvester-cli/issues

    Resources

    The project is written in Go, and using client-go the Kubernetes Go Client libraries to communicate with the Harvester API (which is Kubernetes in fact). Welcome contributions are:

    • Testing it and creating issues
    • Documentation
    • Go code improvement

    What you might learn

    Harvester CLI might be interesting to you if you want to learn more about:

    • GitHub Actions
    • Harvester as a SUSE Product
    • Go programming language
    • Kubernetes API