Kubernetes API caching layer according to Stable Diffusion

Make it faster!

There are use cases that put the Kubernetes API under heavy load - using Rancher at scale can be one of them.

Also, there are use cases in which a connection to the Kubernetes API might not always be present, or with good bandwidth - using Rancher for edge use cases can be one of them.

This project aims to create a local cache serving data from the Kubernetes API - with good performance and displaying last-good-results on a flaky connection.

Goal for this Hackweek

Implement Proof-Of-Concept client-go components backed by SQLite.

https://github.com/moio/vai

Resources

Golang and ideally Kubernetes hackers are more than welcome!

Looking for hackers with the skills:

kubernetes k8s api golang go performance testautomation scalability

This project is part of:

Hack Week 22

Activity

  • over 2 years ago: lizhang liked this project.
  • over 2 years ago: moio added keyword "k8s" to this project.
  • over 2 years ago: moio added keyword "api" to this project.
  • over 2 years ago: moio added keyword "golang" to this project.
  • over 2 years ago: moio added keyword "go" to this project.
  • over 2 years ago: moio added keyword "performance" to this project.
  • over 2 years ago: moio added keyword "testautomation" to this project.
  • over 2 years ago: moio added keyword "scalability" to this project.
  • over 2 years ago: moio added keyword "kubernetes" to this project.
  • over 2 years ago: moio liked this project.
  • over 2 years ago: paulgonin liked this project.
  • over 2 years ago: moio started this project.
  • over 2 years ago: moio originated this project.

  • Comments

    • moio
      over 2 years ago by moio | Reply

      Day 1 question: is a separate daemon design better than creating an Informer backed by a SQL cache.Store?

    • moio
      over 2 years ago by moio | Reply

      Day 1 answer: no. Pivoting project to the creation of a SQL-based Indexer

    • moio
      over 2 years ago by moio | Reply

      Day 2 progress: SQL-backed Store works. https://github.com/moio/vai

    • moio
      over 2 years ago by moio | Reply

      Day 3 progress: SQL-backed Indexer works

    • moio
      over 2 years ago by moio | Reply

      Day 4 question: where would it fit best? Steve or Lasso, and where?

    • moio
      over 2 years ago by moio | Reply

      Day 4 answer: Steve, as an alternative to the current LRU cache of k8s API responses

    • moio
      over 2 years ago by moio | Reply

      Day 5 progress: SQL-backed ThreadSafeStore works. History-preserving VersionedStore also works

    Similar Projects

    terraform-provider-feilong by e_bischoff

    Project Description

    People need to test operating systems and applications on s390 platform.

    Installation from scratch solutions include:

    • just deploy and provision manually add-emoji (with the help of ftpboot script, if you are at SUSE)
    • use s3270 terminal emulation (used by openQA people?)
    • use LXC from IBM to start CP commands and analyze the results
    • use zPXE to do some PXE-alike booting (used by the orthos team?)
    • use tessia to install from scratch using autoyast
    • use libvirt for s390 to do some nested virtualization on some already deployed z/VM system
    • directly install a Linux kernel on a LPAR and use kvm + libvirt from there

    Deployment from image solutions include:

    • use ICIC web interface (openstack in disguise, contributed by IBM)
    • use ICIC from the openstack terraform provider (used by Rancher QA)
    • use zvm_ansible to control SMAPI
    • connect directly to SMAPI low-level socket interface

    IBM Cloud Infrastructure Center (ICIC) harnesses the Feilong API, but you can use Feilong without installing ICIC, provided you set up a "z/VM cloud connector" into one of your VMs following this schema.

    What about writing a terraform Feilong provider, just like we have the terraform libvirt provider? That would allow to transparently call Feilong from your main.tf files to deploy and destroy resources on your system/z.

    Other Feilong-based solutions include:

    • make libvirt Feilong-aware
    • simply call Feilong from shell scripts with curl
    • use zvmconnector client python library from Feilong
    • use zthin part of Feilong to directly command SMAPI.

    Goal for Hackweek 23

    My final goal is to be able to easily deploy and provision VMs automatically on a z/VM system, in a way that people might enjoy even outside of SUSE.

    My technical preference is to write a terraform provider plugin, as it is the approach that involves the least software components for our deployments, while remaining clean, and compatible with our existing development infrastructure.

    Goals for Hackweek 24

    Feilong provider works and is used internally by SUSE Manager team. Let's push it forward!

    Let's add support for fiberchannel disks and multipath.

    Possible goals for Hackweek 25

    Modernization, maturity, and maintenance.