Description

Harvester teams use Harvester seeder in our lab for dev / qa work.

Seeder leverages tinkerbell and is setup to pxe boot our existing lab infra into Harvester clusters.

The provisioning works fine and has improved our release velocity by allowing us to dev/qa changes quicker.

Over the course of time we have encountered the following limitations * Provisioning can be slow, since the multiple reboot/ POST cycles as part of the install process can take a majority of our provisioning time * Resource contention as we have a large number of nodes but with multiple dev/qa streams in flight at any point in time

Goals

The hardware we leverage has lot actual utilisation since a node is likely to be running only a few VM's for actual harvester development QA.

Last hackweek with the introduction of (kubevirtbmc)[https://github.com/starbops/kubevirtbmc], we can manage VM's similar to baremetal by leveraging ipmi based api calls to trigger reboots, set boot order etc. We have already been simulating baremetal as VM's in seeder to allow us to use tinkerbell to ipxe boot VM's and trigger harvester installation.

The goal of this hackweek is to enhance seeder to do the following * add ability to use existing Harvester clusters running kubevirtbmc to create VM's which can be managed as a baremetal node * add ability to create nested Harvester clusters using capability mentioned above.

With a combination of these changes we can further enhance our dev/qa work as we are no longer limited by POST times.

We also improve our resource utilisation as one existing Harvester cluster can run multiple nested Harvester clusters

Resources