SUSE Hack Week: Package MONAI Machine Learning Models for Medical Applications

Project Description

MONAI Deploy aims to become the de-facto standard for developing packaging, testing, deploying, and running medical AI applications in clinical production. MONAI Deploy creates a set of intermediate steps where researchers and physicians can build confidence in the techniques and approaches used with AI — allowing for an iterative workflow.

Contributors to MONAI include Nvidia, Mayo Clinic, King's College London, NHS, Standford University, ... and many others.

The core piece of MONAI Deploy is the Monai Application Package (MAP), which contains the ML model in a "runnable state". It is implemented as a container, and built with docker-nvidia. This is relevant because you need a GPU to build the MAP. You'll see later why this is relevant. This container can be later run on a k8s cluster.

Before you create a MAP, you need to train the ML models. They are trained with MONAI Core, another framework which is a piece of the whole MONAI "puzzle". Those models can be published in the MONAI Model Zoo. They are published using a very specific format, which is called "a bundle".

In the monai deploy app sdk project in github, you can see several examples on how to package a "model bundle" into a MAP. Plus in the documentation you can find a step by step guide on how to build them, meaning how to create the MAP (the container). Examples, both code and documentation, use the ML models in the MONAI model Zoo.

MONAI model Zoo is free, you can search for models and use them for your research. However, there is not such a repository for MAPs, even the docs and examples show how to build those models into MAPS.

And this is the motivation of the project, to create this "link", and release into a registry, at least one MAP based on a model in MONAI Model Zoo.

Goal for this Hackweek

The specific goal is to implement a Continuous Integration workflow that builds a MAP (Monai Application Package), based on the example in code and documentation. Specifically, it is to implement a github action workflow that releases it into github container registry.

Implementation

The github action workflow will be added to a fork of the monai-deploy-app-sdk project, given we will be using the code that is already in the examples directory. Later a Pull Request can be created to the upstream project.

A limitation of this project is that we need to run the github action in a GPU node. ASFAIK github does not support that, so we need to run this on an external runner. For that I will be using MS Azure cloud to host a vm with GPU. For 3 reasons: first, it should be faster to clone from github from azure; second, I will try to use the free 90 days; third, I want to get familiar with Azure.

Finally, most probably I will use terraform to deploy the node in Azure.

This way, every time we want to release a new model in the MAP format, we will deploy a vm in Azure, do the build with the GPU, release into the github container registry, and remove the vm.

Resources

https://monai.io/ https://monai.io/model-zoo.html https://docs.monai.io/projects/monai-deploy-app-sdk/en/latest/gettingstarted/tutorials/monaibundle_app.html https://github.com/Project-MONAI/monai-deploy-app-sdk/tree/main/examples/apps

Looking for hackers with the skills:

ml mlops ai artificial-intelligence gpu azure cloud monai medical containers github_actions github-ci ci

This project is part of:

Hack Week 23

Activity

about 1 year ago: jordimassaguerpla joined this project.

about 1 year ago: vliaskovitis liked this project.

about 1 year ago: rtsvetkov started this project.

about 1 year ago: jordimassaguerpla added keyword "github_actions" to this project.

about 1 year ago: jordimassaguerpla added keyword "github-ci" to this project.

about 1 year ago: jordimassaguerpla added keyword "ci" to this project.

about 1 year ago: jordimassaguerpla added keyword "gpu" to this project.

about 1 year ago: jordimassaguerpla added keyword "azure" to this project.

about 1 year ago: jordimassaguerpla added keyword "cloud" to this project.

about 1 year ago: jordimassaguerpla added keyword "monai" to this project.

about 1 year ago: jordimassaguerpla added keyword "medical" to this project.

about 1 year ago: jordimassaguerpla added keyword "containers" to this project.

about 1 year ago: jordimassaguerpla added keyword "ml" to this project.

about 1 year ago: jordimassaguerpla added keyword "mlops" to this project.

about 1 year ago: jordimassaguerpla added keyword "ai" to this project.

about 1 year ago: jordimassaguerpla added keyword "artificial-intelligence" to this project.

about 1 year ago: jordimassaguerpla originated this project.

Comments

about 1 year ago by jordimassaguerpla | Reply

I was able to create a terraform file and a workflow file but then I was not able to make the build work.

Here the terraform file:

https://github.com/jordimassaguerpla/monai-deploy-app-sdk/blob/main/main.tf

Here the workflow file:

https://github.com/jordimassaguerpla/monai-deploy-app-sdk/blob/main/.github/workflows/buildandpush_models.yml

about 1 year ago by jordimassaguerpla | Reply

I think the issue is that it tries to load the container, but I had not installed nvidia-docker2, and thus it can't load the container.

12 months ago by jordimassaguerpla | Reply

Here the fix for libseccomp, so nvidia-container-toolkit can be installed: https://build.opensuse.org/request/show/1128309 Here the fix for nvidia-holoscan: https://github.com/nvidia-holoscan/holoscan-sdk/pull/14

With these 2 fixes and by increasing the Disc in Azure to 64GB, I was able to build the ML model as a container :)

12 months ago by jordimassaguerpla | Reply

And voilà, here the MONAI Application Package ready to be used:

https://github.com/jordimassaguerpla/monai-deploy-app-sdk/pkgs/container/monai-deploy-app-sdk%2Fsimple_app-x64-workstation-dgpu-linux-amd64