image-tools: simple tool for mirror/save/load container images & KDM and chart image list generator.
an invention by StarryWang
Updated over 2 years ago. 3 hacker ♥️. 2 followers.

Project Description

Currently, the way to install Rancher in Air-Gap mode (with personal registry server) is using the rancher-save/load-images.sh script to save container images required by Rancher into tar.gz tarball and load it into the personal registry. These scripts work fine when saving and loading single-arch images.

However, if we need to install Rancher cluster in AMD64 and ARM64 architecture, we need a tool to mirror multi-arch images from public registry to personal registry or save images into tarball and load it into personal registry (when no network connection). So this project is mainly used to mirror/save and load multi-arch container images from the public registry to the personal registry (by using skopeo) and build manifest list (by using docker-buildx). I also implemented validating functions to ensure all container images were mirrored/loaded into the destination registry.

Currently, this project has already finished the mirror/load/save and validation functions, and I am developing the new functions to generate an upgrade image list from KDM JSON data and chart repos during this HackWeek.

This tool is written in Go and the compiled binary file can be found on the GitHub Release page. And this tool also provides container image for mirror images in CI pipeline automatically.

Goal for this Hackweek

Here are the things I'm going to do during HackWeek 22.

  1. Implement the functions of generating an image list from KDM JSON data and chart repos.
  2. Add English documents for this project.

Resources

  1. image-tools:
  2. skopeo:
  3. KDM (kontainer-driver-metadata):
  4. Collect and Publish Images to your Private Registry:

Looking for hackers with the skills:

rancher containers skopeo kdm

This project is part of:

Hack Week 22

Activity

  • over 2 years ago: danishprakash joined this project.
  • over 2 years ago: paulgonin liked this project.
  • over 2 years ago: danishprakash liked this project.
  • over 2 years ago: StarryWang started this project.
  • over 2 years ago: StarryWang liked this project.
  • over 2 years ago: StarryWang added keyword "rancher" to this project.
  • over 2 years ago: StarryWang added keyword "containers" to this project.
  • over 2 years ago: StarryWang added keyword "skopeo" to this project.
  • over 2 years ago: StarryWang added keyword "kdm" to this project.
  • over 2 years ago: StarryWang originated this project.


    • Comments

    • StarryWang
      over 2 years ago by StarryWang | Reply

      The generate-list sub-command is available in the v1.4.0-rc2

      I'll make a final release when it becomes stable after this hackweek.

      English docs have been supplemented by @danishprakash .

    • StarryWang
      over 2 years ago by StarryWang | Reply

      Name needed: the name of this project image-tools is not good enough, can someone help me with a more interesting name of this project?

    • StarryWang
      over 2 years ago by StarryWang | Reply

      Just released 1.4.0-rc3 and made some improvements on the generate-list sub-command, this HackWeek project can be marked as finished.

    Similar Projects

    A CLI for Harvester by mohamed.belgaied

    Harvester does not officially come with a CLI tool, the user is supposed to interact with Harvester mostly through the UI. Though it is theoretically possible to use kubectl to interact with Harvester, the manipulation of Kubevirt YAML objects is absolutely not user friendly. Inspired by tools like multipass from Canonical to easily and rapidly create one of multiple VMs, I began the development of Harvester CLI. Currently, it works but Harvester CLI needs some love to be up-to-date with Harvester v1.0.2 and needs some bug fixes and improvements as well.

    Project Description

    Harvester CLI is a command line interface tool written in Go, designed to simplify interfacing with a Harvester cluster as a user. It is especially useful for testing purposes as you can easily and rapidly create VMs in Harvester by providing a simple command such as: harvester vm create my-vm --count 5 to create 5 VMs named my-vm-01 to my-vm-05.

    asciicast

    Harvester CLI is functional but needs a number of improvements: up-to-date functionality with Harvester v1.0.2 (some minor issues right now), modifying the default behaviour to create an opensuse VM instead of an ubuntu VM, solve some bugs, etc.

    Github Repo for Harvester CLI: https://github.com/belgaied2/harvester-cli

    Done in previous Hackweeks

    • Create a Github actions pipeline to automatically integrate Harvester CLI to Homebrew repositories: DONE
    • Automatically package Harvester CLI for OpenSUSE / Redhat RPMs or DEBs: DONE

    Goal for this Hackweek

    The goal for this Hackweek is to bring Harvester CLI up-to-speed with latest Harvester versions (v1.3.X and v1.4.X), and improve the code quality as well as implement some simple features and bug fixes.

    Some nice additions might be: * Improve handling of namespaced objects * Add features, such as network management or Load Balancer creation ? * Add more unit tests and, why not, e2e tests * Improve CI * Improve the overall code quality * Test the program and create issues for it

    Issue list is here: https://github.com/belgaied2/harvester-cli/issues

    Resources

    The project is written in Go, and using client-go the Kubernetes Go Client libraries to communicate with the Harvester API (which is Kubernetes in fact). Welcome contributions are:

    • Testing it and creating issues
    • Documentation
    • Go code improvement

    What you might learn

    Harvester CLI might be interesting to you if you want to learn more about:

    • GitHub Actions
    • Harvester as a SUSE Product
    • Go programming language
    • Kubernetes API
    • Kubevirt API objects (Manipulating VMs and VM Configuration in Kubernetes using Kubevirt)

    Cluster API Provider for Harvester by rcase

    Project Description

    The Cluster API "infrastructure provider" for Harvester, also named CAPHV, makes it possible to use Harvester with Cluster API. This enables people and organisations to create Kubernetes clusters running on VMs created by Harvester using a declarative spec.

    The project has been bootstrapped in HackWeek 23, and its code is available here.

    Work done in HackWeek 2023

    • Have a early working version of the provider available on Rancher Sandbox : *DONE *
    • Demonstrated the created cluster can be imported using Rancher Turtles: DONE
    • Stretch goal - demonstrate using the new provider with CAPRKE2: DONE and the templates are available on the repo

    DONE in HackWeek 24:

    DONE in 2025 (out of Hackweek)

    • Support of ClusterClass
    • Add to clusterctl community providers, you can add it directly with clusterctl
    • Testing on newer versions of Harvester v1.4.X and v1.5.X
    • Support for clusterctl generate cluster ...
    • Improve Status Conditions to reflect current state of Infrastructure
    • Improve CI (some bugs for release creation)

    Goals for HackWeek 2025

    • FIRST and FOREMOST, any topic is important to you
    • Add e2e testing
    • Certify the provider for Rancher Turtles
    • Add Machine pool labeling
    • Add PCI-e passthrough capabilities.
    • Other improvement suggestions are welcome!

    Thanks to @isim and Dominic Giebert for their contributions!

    Resources

    Looking for help from anyone interested in Cluster API (CAPI) or who wants to learn more about Harvester.

    This will be an infrastructure provider for Cluster API. Some background reading for the CAPI aspect:

    Rancher/k8s Trouble-Maker by tonyhansen

    Project Description

    When studying for my RHCSA, I found trouble-maker, which is a program that breaks a Linux OS and requires you to fix it. I want to create something similar for Rancher/k8s that can allow for troubleshooting an unknown environment.

    Goals for Hackweek 25

    • Update to modern Rancher and verify that existing tests still work
    • Change testing logic to populate secrets instead of requiring a secondary script
    • Add new tests

    Goals for Hackweek 24 (Complete)

    • Create a basic framework for creating Rancher/k8s cluster lab environments as needed for the Break/Fix
    • Create at least 5 modules that can be applied to the cluster and require troubleshooting

    Resources

    • https://github.com/celidon/rancher-troublemaker
    • https://github.com/rancher/terraform-provider-rancher2
    • https://github.com/rancher/tf-rancher-up
    • https://github.com/rancher/quickstart

    Technical talks at universities by agamez

    Description

    This project aims to empower the next generation of tech professionals by offering hands-on workshops on containerization and Kubernetes, with a strong focus on open-source technologies. By providing practical experience with these cutting-edge tools and fostering a deep understanding of open-source principles, we aim to bridge the gap between academia and industry.

    For now, the scope is limited to Spanish universities, since we already have the contacts and have started some conversations.

    Goals

    • Technical Skill Development: equip students with the fundamental knowledge and skills to build, deploy, and manage containerized applications using open-source tools like Kubernetes.
    • Open-Source Mindset: foster a passion for open-source software, encouraging students to contribute to open-source projects and collaborate with the global developer community.
    • Career Readiness: prepare students for industry-relevant roles by exposing them to real-world use cases, best practices, and open-source in companies.

    Resources

    • Instructors: experienced open-source professionals with deep knowledge of containerization and Kubernetes.
    • SUSE Expertise: leverage SUSE's expertise in open-source technologies to provide insights into industry trends and best practices.