Project Description
Provide image based transactional updates for MicroOS.
In the embedded world distributors (not customers) prefer image based updates, as they claim they are not able to test so many small updates (which sounds more like an excuse to not invest in test automation and not increase the automatic test coverage).
There are many solutions for image based updates (Mender, RAUC, SWUpdate, Swupd, ...), but they all require an A/B partition setup, thus are limited in the number of parallel installed systems, systems to rollback too, and the size a system can grow.
This project will try to provide a PoC to update openSUSE MicroOS (Container Host system role) via images, by combining existing tools (like kiwi, RAUC, ...) to create update-images, transfer them to a host (USB-Stick, OTA, container, ...) and apply as a new snapshot like transactional-update would update the system with zypper. Use all the tooling around it (btrfs tools, snapper, health-checker, ...), as they exist already, are proven to work and reduces the required changes to the system.
Goal for this Hackweek
- Build tar archives with kiwi of openSUSE MicroOS "Container Host"
- Use RAUC to create a verity bundle
- Transfer the bundle via USB Stick, OTA and/or container to the client.
- Follow the steps transactional-update would do by creating a new snapshot, install the image (use rsync or something similar, to not duplicate blocks, else de-duplicate) and make the new image the new default if everything is correct
- Reboot into the new image
Problems to solve
- Integrate verity bundle building into OBS
- How to create new users and groups used in the new image?
- How to handle updating of /etc? UsrEtc is one solution. There is no package manager to update/merge configuration files, but there are %pre and %post scripts accessing /etc and /var
- How to handle modifications to /var?
- At minimum update-alternative and selinux-policy install in /var. Only /boot and /usr (after UsrMove) are allowed, else /bin, /sbin/, /lib and /lib64.
Resources
Updates
Day1
- rauc is in the current form not useable, but the idea and parts of the code can be re-used
- There is an openSUSE MicroOS Container Host qcow2 image for testing. No transactional-update nor zypper, but rpm due to dependencies of suse-module-tools. Layout is standard MicroOS with /etc as overlay
- There is a PoC script to build squashfs images containing the update images in casync format
- There is a PoC script to apply the update to an btrfs snapshot, build the initrd and bootloader configuration and boot it.
- casync has same serious bugs, as result an update of an 550MB image with the same image requires over 300MB disk space and not only a few bytes for the reflinks. Additional, it is not possible to package the data from a xfs partition and deploy it on a btrfs partition (and most likely vice versa), the handling of the '.' entry looks broken.
The biggest problem is the /etc directory. /etc cannot be part of the image, but I was not able to build a working image with /etc in a subvolume. Using the usrMove result would allow to have /etc on a read-write root filesystem and /boot plus /usr as image. But in this case, root would be no longer read-only (has advantages and disadvantages) and rollback is impossible. Putting /boot on /usr is not possible, as we have to generate initrd and files for the bootloader.
Day2
- Start writing "tiu create" in C. Missing pieces:
- Cleanup the subvolumes before packaging
- Add verity checksum
Day3
was not such successful, but * Added dm-verity checksum to squashfs archive, but not yet signed (so not secure) * Start writing "tiu extract".
Day4
- Verification of tiu archive with dm-vertiy works, only signing is missing
- "tiu extract" can verify, mount and extract an tiu archive to a btrfs subvolume
- Many bug fixes
Day5
- Finish implementation of downloading the tiu archive
- Write documentation
- Start working on signing of the tiu archives
Looking for hackers with the skills:
Nothing? Add some keywords!
This project is part of:
Hack Week 20
Activity
