The goal is to run the Elastic Inference Demo managed by kubernetes on a Raspberry Pi4 Cluster with openSUSE Kubic.

• The openSUSE Kubic image should configure itself correct at the first boot with ignition.
• The demo should run without network connection, so a proxy is needed:
  • Two network interfaces, running openSUSE Kubic as Gateway
  • IP address of network interfaces is shown on a LCD display
  • All workload should run in containers:
    • local registry with authentication: everybody can pull, only admins can push
    • dhcp server for the other Raspberry PIs to setup the network
    • name server
    • proxy to access the services from outside
  • KubicD and a salt-master to manage kubernetes and the nodes
• Kubernetes Master
• Kubernetes Worker with a Neural Compute Stick (NCS)
• Kubernetes Worker

Main challenge is: the inference container needs to be able to access the NCS via USB, kubernetes needs to schedule the inference container on that host, only one inference container is allowed to be scheduled on this host.

Update:

2020-02-11

• We have working images with ignition for RPi4: https://download.opensuse.org/repositories/home:/kukuk:/raspi:/images/openSUSEFactoryARM/
• We have two ignition configuration files:
  • First to setup password/ssh/LCD display of the first RPi4.
  • Second to setup password/ssh/salt-minion of the other RPi4's.
• The "gateway" RPi4:
  • runs the ContainerHost image
  • has two NICs and a LCD display. The display shows the internal and external IP of the network interfaces.
  • runs registry, bind and dhcp as container. haproxy is WiP
  • runs salt-master, kubicd and haproxy
  • has a RTC and runs chrony to provide a NTP service to the internal net (other RPi4s)
• The other three RPi4's:
  • run the Kubeadm image
  • are salt-minions
  • will be configured remotely via kubicd/salt
  • have no network access, they can only fetch the time and container images from the "gateway"
• kubicctl/kubicd is able:
  • deploy a kubernetes master and add nodes via salt
  • deploy metallb
  • deploy hello-kubic
• The "internet" can access hello-kubic via haproxy
• The Elastic Inference Demo can be deployed, but the CPUs are too slow for this, they manage only 0.04-0.08 FPS.
• Next steps: get the Intel Neural Compute 2 Stick (NCS2) working and accessible out of the kubernetes container.

2020-02-12

• The Intel Neural Compute Stick 2 (NCS2) will be booted during the boot of the machine or can be booted later with help of the myriadctl program written by me for this case
• A container running in the kubernetes cluster can use the NCS2 if it run on the right host (missing: kubernetes resource handling)
• The openvino kubernetes deployment can classify pictures with help of the NCS2
• Results of the dldt benchmark running in a container in kubernetes:
  • With NCS2: throughput: 263.34 FPS
  • Without: throughput: 7.55 FPS
• The intel python code for elastic inference demo is core dumping after loading the model to the stick ... :(
• haproxy runs now in a container, too

2020-02-13

• The dldt python bindings are broken: not using the CPU plugin will lead to a crash. The C++ code is working fine, but the elastic inference demo is written in python ...
• Enhance openvino-k8s-yaml: classification deloyment uses now kustomize and requirement of a NCS2 device can be added
• The openvino classification container runs now only on the node with a NCS2 attached

2020-02-14

• At least the openvino container is running and working, so you can classify pictures.

Documentation

Github project with more detailed instructions and configuration files.

Presentation