Modular kernel packaging
a project by mwilck
Updated over 2 years ago. 7 hacker ♥️. 4 followers.

Project Description

Create a PoC for a modular packaged kernel. This means:

  • the kernel package itself is minimal
  • all device drivers are packaged as KMPs (kernel module packages)
  • some other modules (e.g. subsystems like netfilter, SCSI, …) are packaged as KMPs, too
  • module dependencies (depmod) translate into package dependencies and are handled by rpm
  • hardware requirements are handled by zypper/YaST via modalias
  • other system requirements (e.g. "this system needs a firewall") would be handled by patterns

Advantages

  • minimal installation size and download size for kernel and drivers — only actually required/used modules are installed
  • reduced build time: drivers don't need to be rebuilt with every kernel update, only if the driver itself (or the APIs it uses) are updated. KABI tracking ensures that drivers remain compatible with updated kernels.
  • vice-versa, a driver update doesn't necessitate a full kernel update.
  • this might also simplify QA procedures: driver testing could be separated from core kernel testing.
  • replacement of in-box drivers by out-of-box drivers simplified; it would become a routine procedure
  • conservative customers can update components selectively, reducing regression risks
  • package management techniques (e.g. download statistics) can be used to track usage of drivers and identify modules to be disabled
  • we could get rid of package bundles like kernel-default-base or kernel-default-extra.

Disadvantages / Challenges

  • "Missing driver problem": e.g. network controller breaks, new controller needs new driver → unavoidable, patterns e.g. "kernel-network-driver-all" or "kernel-usb-driver-all" as precaution (but this would forfeit a lot of the size advantage)
  • weak dependencies, or dependencies that aren't expressed in depmod terms. We handle this partially already for initrd (SUSE INITRD mechanism). We must translate it into package dependencies
  • user space dependencies: many tools might not be prepared for a large number of missing drivers in an (open)SUSE environment. Need to look into dracut etc.
  • is our KABI tracking good enough? So far it works well for the small number of KMPs, but what if we get 1000s of KMPs with complex dependencies?
  • Does the KABI dependency tracking scale to this level, or will dependency checks be extremely slow?
  • It gets more difficult to assess system supportablity. Tooling is required to verify that installed driver packages are compatible.
  • we can't run depmod after every KMP installation, it'd be far too slow (I have a patched depmod available that would make this possible with "incremental" mode).
  • avoid excessive meta data duplication (kernel changelog in every KMP)
  • the KMP concept needs to be revised. The multiversion(kernel) property of KMPs allows to install the same KMP for different KABI versions, but doesn't work well for version updates of the KMP itself. That would be a big problem, because we update some drivers frequently.
  • do we need a means for kernel module management tools (modprobe) to hook into package management (zypper) for pulling in missing modules automatically?
  • …

Goal for this Hackweek

  • Make it possible to build "real" KMPs from the kernel package build. Note that the in-kernel -kmp packages such as dlm-kmp aren't "real" KMPs; they use a hard kernel version dependency rather than KABI dependencies.
  • Hack on the kernel build scripts to enable modular packaging. Perhaps one-KMP-per-module as a first step, to experiment with module interdependencies. 2nd step would be combining modules into reasonable sets. Alternatively, start with a set of modules to be packaged as KMPs (e.g. device drivers) and increase the set step-by-step.
  • Experiment with practical use of the modular kernel. Figure out problems with the interaction of module and package management. Play through the "missing driver" scenario, and try to find a concept for aworkaround.

Later goals

  • Develop a modified KMP concept. Make version of the KABI the KMP was compiled against part of the package name rather than the version (instead of foo-kmp-default-1.0_k5.3.18_59.5, use foo-kmp-default-5.3.18_59.5-1.0, indicating module foo for the KABI of 5.3.18_59.5, version 1.0)
  • Tackle a few of the challenges above
  • Explore possibilities to improve packaging of kernel firmware, too. Currently FW is packaged in big chunks by vendor. Packaging individual drivers might facilitate fine-grained dependencies (driver pulls in required FW packages).

Join this project

Looking for hackers with the skills:

kmp kernel module rpm zypper packagemanagement kabi

This project is part of:

Hack Week 21 Hack Week 22

Activity

  • over 2 years ago: mkoutny liked this project.
  • over 2 years ago: mwilck started this project.
  • over 2 years ago: ptesarik liked this project.
  • over 2 years ago: mbrugger liked this project.
  • over 2 years ago: sbahling liked this project.
  • over 2 years ago: llansky3 liked this project.
  • over 2 years ago: coolgw liked this project.
  • over 2 years ago: avicenzi liked this project.
  • over 2 years ago: mwilck added keyword "kabi" to this project.
  • over 2 years ago: mwilck added keyword "packagemanagement" to this project.
  • over 2 years ago: mwilck added keyword "zypper" to this project.
  • over 2 years ago: mwilck added keyword "rpm" to this project.
  • over 2 years ago: mwilck added keyword "module" to this project.
  • over 2 years ago: mwilck added keyword "kernel" to this project.
  • over 2 years ago: mwilck added keyword "kmp" to this project.
  • over 2 years ago: mwilck originated this project.


    • Comments

    • mwilck
      over 2 years ago by mwilck | Reply

      Jun 27

      I spent the day studying the kernel build scripts. Using them and understanding them are two different things. I'll try to use a similar approach as kernel-default-base: Just repackage existing modules. This will speed up development, as I'll avoid long build times for the kernel package. OBS repos created under home:mwilck:ModularKernel (not much to see yet).

    • mwilck
      over 2 years ago by mwilck | Reply

      Jun 28

      I have a package with just the kernel image now, and at least one auto-built KMP. I am using a variant of the kernel-default-base package to simply repackage stuff that's currently built in the default package.

    • mwilck
      over 2 years ago by mwilck | Reply

      Jun 29 — Jul 1

      Packaging of both the kernel image and modules works. I need to transform the scripts from kernel-default-base into rpm macros, but that's not a big deal, actually.

      Unfortunately I couldn't make more progress this week due to sickness.

    • mwilck
      over 2 years ago by mwilck | Reply

      Later

      The kernel-default-image package now packages the whole set of modules from kernel-default-base, each module packaged separately. No actual compilation is done, the compiled modules are just taken from kernel-default, the same way as it's done in kernel-default-base. If we ever want to change this, the real challenge will be how to determine the list of subpackages, as it's currently calculated dynamically from modules.dep. We would basically need to generate the list of subpackages from the list of .ko files built during kernel compilation. I am not sure if that's possible. If not, we will have to do at least one "normal" build for every config to derive the module list first before modularly packaged kernels can be built. But we have a long way to go until then, anyway.

      Dependency resolution works the same way as for kernel-default-base; thus it's sufficient to list a set of desired modules or module-name-matching regular expressions, and the scripts will figure out the full set to satisfy all requirements.

      The module packages have ksym() requires and (if they export anything) also ksym() provides, as well as modalias() supplements.

      > rpm -qp --supplements ehci-pci-kmp-default-5.14.21-lp154.23.1.g49db222.0.x86_64.rpm modalias(kernel-default:pci:v*d*sv*sd*bc0Csc03i20*) modalias(kernel-default:pci:v0000104Ad0000CC00sv*sd*bc*sc*i*) > rpm -qp --requires ehci-pci-kmp-default-5.14.21-lp154.23.1.g49db222.0.x86_64.rpm coreutils grep kernel-default-image ksym(default:__dynamic_dev_dbg) = 22868a41 ksym(default:__fentry__) = bdfb6dbb ksym(default:__pci_register_driver) = c2126345 ksym(default:__stack_chk_fail) = d0da656b ksym(default:_dev_info) = fe960afa ksym(default:_dev_warn) = 5cba8a10 ksym(default:device_set_wakeup_capable) = 226a1ef2 ksym(default:dma_set_coherent_mask) = 93f12bc3 ksym(default:ehci_init_driver) = d72c5993 ksym(default:ehci_resume) = 645790fb ... > rpm -qp --provides ehci-hcd-kmp-default-5.14.21-lp154.23.1.g49db222.0.x86_64.rpm ehci-hcd-kmp = 5.14.21 ehci-hcd-kmp-default = 5.14.21-lp154.23.1.g49db222.0 ehci-hcd-kmp-default(x86-64) = 5.14.21-lp154.23.1.g49db222.0 kmod(ehci_hcd.ko) ksym(default:ehci_adjust_port_wakeup_flags) = e6bf909e ksym(default:ehci_handshake) = 1456664e ksym(default:ehci_hub_control) = 913627e8 ksym(default:ehci_init_driver) = d72c5993 ksym(default:ehci_reset) = 145925dc ksym(default:ehci_resume) = 645790fb ksym(default:ehci_setup) = 4048a7e9 ksym(default:ehci_suspend) = 2c94b03f The build log is interesting, too.

      I had to modify the scripts looking for provides and supplements for this to work. Thus I'm shipping these as part of the package.

      Not surprisingly, the package repo contains a long list of packages. Including modules from -extra, I built a set of almost 3000 packages. Not quite texlive, but getting close add-emoji

      I had to stop at this point; so I haven't done any installation tests with the KMPs generated this way yet. The dependencies look quite sane though. What's obviously missing at this point is some patterns to allow installing e.g. all netfilter modules. For drivers, modalias should work (zypper inr --no-recommends), but I havent't tested it. The repo also urgently needs my performance fixes for depmod from last year, which are unfortunately still unfinished.

    • michals
      over 2 years ago by michals | Reply

      The problem with this approach is that when you include the dependencies the subpackages may overlap, and while rpm itself would be fine with that because the modules coming from one kernel build are identical the packaging policy forbids it.

      • tiwai
        over 2 years ago by tiwai | Reply

        What do you mean by "the dependencies the subpackages may overlap"? The same ksym() checksum provided by multiple kmps (of various versions)?

        I guess a partial solution would be to embed the hard dependency from the result of depmod. e.g. if a module has dependency on another module abc (as found in depmod), we can put "Requires: abc-kmp-default = %{version}" explicitly at the build time. Also, a softdep could be interpreted with Supplements or such, too.

    Similar Projects

    Modularization and Modernization of cifs.ko for Enhanced SMB Protocol Support by hcarvalho

    Creator:
    Enzo Matsumiya ematsumiya@suse.de @ SUSE Samba team
    Members:
    Henrique Carvalho henrique.carvalho@suse.com @ SUSE Samba team

    Description

    Split cifs.ko in 2 separate modules; one for SMB 1.0 and 2.0.x, and another for SMB 2.1, 3.0, and 3.1.1.

    Goals

    Primary

    Start phasing out/deprecation of older SMB versions

    Secondary

    • Clean up of the code (with focus on the newer versions)
    • Update cifs-utils
    • Update documentation
    • Improve backport workflow (see below)

    Technical details

    Ideas for the implementation.

    • fs/smb/client/{old,new}.c to generate the respective modules
      • Maybe don't create separate folders? (re-evaluate as things progresses!)
    • Remove server->{ops,vals} if possible
    • Clean up fs_context.* -- merge duplicate options into one, handle them in userspace utils
    • Reduce code in smb2pdu.c -- tons of functions with very similar init/setup -> send/recv -> handle/free flow
    • Restructure multichannel
      • Treat initial connection as "channel 0" regardless of multichannel enabled/negotiated status, proceed with extra channels accordingly
      • Extra channel just point to "channel 0" as the primary server, no need to allocate an extra TCPServerInfo for each one
    • Authentication mechanisms
      • Modernize algorithms (references: himmelblau, IAKERB/Local KDC, SCRAM, oauth2 (Azure), etc.

    Create DRM drivers for VESA and EFI framebuffers by tdz

    Description

    We already have simpledrm for firmware framebuffers. But the driver is originally for ARM boards, not PCs. It is already overloaded with code to support both use cases. At the same time it is missing possible features for VESA and EFI, such as palette modes or EDID support. We should have DRM drivers for VESA and EFI interfaces. The infrastructure exists already and initial drivers can be forked from simpledrm.

    Goals

    • Initially, a bare driver for VESA or EFI should be created. It can take functionality from simpledrm.
    • Then we can begin to add additional features. The boot loader can provide EDID data. With VGA hardware, VESA can support paletted modes or color management. Example code exists in vesafb.

    Contributing to Linux Kernel security by pperego

    Description

    A couple of weeks ago, I found this blog post by Gustavo Silva, a Linux Kernel contributor.

    I always strived to start again into hacking the Linux Kernel, so I asked Coverity scan dashboard access and I want to contribute to Linux Kernel by fixing some minor issues.

    I want also to create a Linux Kernel fuzzing lab using qemu and syzkaller

    Goals

    1. Fix at least 2 security bugs
    2. Create the fuzzing lab and having it running

    The story so far

    • Day 1: setting up a virtual machine for kernel development using Tumbleweed. Reading a lot of documentation, taking confidence with Coverity dashboard and with procedures to submit a kernel patch
    • Day 2: I read really a lot of documentation and I triaged some findings on Coverity SAST dashboard. I have to confirm that SAST tool are great false positives generator, even for low hanging fruits.
    • Day 3: Working on trivial changes after I read this blog post: https://www.toblux.com/posts/2024/02/linux-kernel-patches.html. I have to take confidence with the patch preparation and submit process yet.
      • First trivial patch sent: using strtruefalse() macro instead of hard-coded strings in a staging driver for a lcd display
      • Fix for a dereference before null check issue discovered by Coverity (CID 1601566) https://scan7.scan.coverity.com/#/project-view/52110/11354?selectedIssue=1601566
    • Day 4: Triaging more issues found by Coverity.
      • The patch for CID 1601566 was refused. The check against the NULL pointer was pointless so I prepared a version 2 of the patch removing the check.
      • Fixed another dereference before NULL check in iwlmvmparsewowlaninfo_notif() routine (CID 1601547). This one was already submitted by another kernel hacker :(
    • Day 5: Wrapping up. I had to do some minor rework on patch for CID 1601566. I found a stalker bothering me in private emails and people I interacted with me, advised he is a well known bothering person. Markus Elfring for the record.

    • Wrapping up: being back doing kernel hacking is amazing and I don't want to stop it. My battery pack is completely drained but changing the scope gave me a great twist and I really want to feel this energy not doing a single task for months.

      I failed in setting up a fuzzing lab but I was too optimistic for the patch submission process.

    The patches

    1

    RISC-V emulator in GLSL capable of running Linux by favogt

    Description

    There are already numerous ways to run Linux and some programs through emulation in a web browser (e.g. x86 and riscv64 on https://bellard.org/jslinux/), but none use WebGL/WebGPU to run the emulation on the GPU.

    I already made a PoC of an AArch64 (64-bit Arm) emulator in OpenCL which is unfortunately hindered by a multitude of OpenCL compiler bugs on all platforms (Intel with beignet or the new compute runtime and AMD with Mesa Clover and rusticl). With more widespread and thus less broken GLSL vs. OpenCL and the less complex implementation requirements for RV32 (especially 32bit integers instead of 64bit), that should not be a major problem anymore.

    Goals

    Write an RISC-V system emulator in GLSL that is capable of booting Linux and run some userspace programs interactively. Ideally it is small enough to work on online test platforms like Shaderoo with a custom texture that contains bootstrap code, kernel and initrd.

    Minimum:

    riscv32 without FPU (RV32 IMA) and MMU (µClinux), running Linux in M-mode and userspace in U-mode.

    Stretch goals:

    FPU support, S-Mode support with MMU, SMP. Custom web frontend with more possibilities for I/O (disk image, network?).

    Resources

    RISC-V ISA Specifications
    Shaderoo
    OpenGL 4.5 Quick Reference Card

    Result as of Hackweek 2024

    WebGL turned out to be insufficient, it only supports OpenGL ES 3.0 but imageLoad/imageStore needs ES 3.1. So we switched directions and had to write a native C++ host for the shaders.

    As of Hackweek Friday, the kernel attempts to boot and outputs messages, but panics due to missing memory regions.

    Since then, some bugs were fixed and enough hardware emulation implemented, so that now Linux boots with framebuffer support and it's possible to log in and run programs!

    The repo with a demo video is available at https://github.com/Vogtinator/risky-v

    early stage kdump support by mbrugger

    Project Description

    When we experience a early boot crash, we are not able to analyze the kernel dump, as user-space wasn't able to load the crash system. The idea is to make the crash system compiled into the host kernel (think of initramfs) so that we can create a kernel dump really early in the boot process.

    Goal for the Hackweeks

    1. Investigate if this is possible and the implications it would have (done in HW21)
    2. Hack up a PoC (done in HW22 and HW23)
    3. Prepare RFC series (giving it's only one week, we are entering wishful thinking territory here).

    update HW23

    • I was able to include the crash kernel into the kernel Image.
    • I'll need to find a way to load that from init/main.c:start_kernel() probably after kcsan_init()
    • I workaround for a smoke test was to hack kexec_file_load() systemcall which has two problems:
      1. My initramfs in the porduction kernel does not have a new enough kexec version, that's not a blocker but where the week ended
      2. As the crash kernel is part of init.data it will be already stale once I can call kexec_file_load() from user-space.

    The solution is probably to rewrite the POC so that the invocation can be done from init.text (that's my theory) but I'm not sure if I can reuse the kexec infrastructure in the kernel from there, which I rely on heavily.

    update HW24

    • Day1
      • rebased on v6.12 with no problems others then me breaking the config
      • setting up a new compilation and qemu/virtme env
      • getting desperate as nothing works that used to work
    • Day 2
      • getting to call the invocation of loading the early kernel from __init after kcsan_init()

    • Day 3

      • fix problem of memdup not being able to alloc so much memory... use 64K page sizes for now
      • code refactoring
      • I'm now able to load the crash kernel
      • When using virtme I can boot into the crash kernel, also it doesn't boot completely (major milestone!), crash in elfcorehdr_read_notes()

    • Day 4

      • crash systems crashes (no pun intended) in copy_old_mempage() link; will need to understand elfcorehdr...
      • call path vmcore_init() -> parse_crash_elf_headers() -> elfcorehdr_read() -> read_from_oldmem() -> copy_oldmem_page() -> copy_to_iter()

    • Day 5

      • hacking arch/arm64/kernel/crash_dump.c:copy_old_mempage() to see if crash system really starts. It does.
      • fun fact: retested with more reserved memory and with UEFI FW, host kernel crashes in init but directly starts the crash kernel, so it works (somehow) \o/

    • TODOs

      • fix elfcorehdr so that we actually can make use of all this...
      • test where in the boot __init() chain we can/should call kexec_early_dump()

    A CLI for Harvester by mohamed.belgaied

    [comment]: # Harvester does not officially come with a CLI tool, the user is supposed to interact with Harvester mostly through the UI [comment]: # Though it is theoretically possible to use kubectl to interact with Harvester, the manipulation of Kubevirt YAML objects is absolutely not user friendly. [comment]: # Inspired by tools like multipass from Canonical to easily and rapidly create one of multiple VMs, I began the development of Harvester CLI. Currently, it works but Harvester CLI needs some love to be up-to-date with Harvester v1.0.2 and needs some bug fixes and improvements as well.

    Project Description

    Harvester CLI is a command line interface tool written in Go, designed to simplify interfacing with a Harvester cluster as a user. It is especially useful for testing purposes as you can easily and rapidly create VMs in Harvester by providing a simple command such as: harvester vm create my-vm --count 5 to create 5 VMs named my-vm-01 to my-vm-05.

    asciicast

    Harvester CLI is functional but needs a number of improvements: up-to-date functionality with Harvester v1.0.2 (some minor issues right now), modifying the default behaviour to create an opensuse VM instead of an ubuntu VM, solve some bugs, etc.

    Github Repo for Harvester CLI: https://github.com/belgaied2/harvester-cli

    Done in previous Hackweeks

    • Create a Github actions pipeline to automatically integrate Harvester CLI to Homebrew repositories: DONE
    • Automatically package Harvester CLI for OpenSUSE / Redhat RPMs or DEBs: DONE

    Goal for this Hackweek

    The goal for this Hackweek is to bring Harvester CLI up-to-speed with latest Harvester versions (v1.3.X and v1.4.X), and improve the code quality as well as implement some simple features and bug fixes.

    Some nice additions might be: * Improve handling of namespaced objects * Add features, such as network management or Load Balancer creation ? * Add more unit tests and, why not, e2e tests * Improve CI * Improve the overall code quality * Test the program and create issues for it

    Issue list is here: https://github.com/belgaied2/harvester-cli/issues

    Resources

    The project is written in Go, and using client-go the Kubernetes Go Client libraries to communicate with the Harvester API (which is Kubernetes in fact). Welcome contributions are:

    • Testing it and creating issues
    • Documentation
    • Go code improvement

    What you might learn

    Harvester CLI might be interesting to you if you want to learn more about:

    • GitHub Actions
    • Harvester as a SUSE Product
    • Go programming language
    • Kubernetes API

    Switch software-o-o to parse repomd data by hennevogel

    Currently software.opensuse.org search is using the OBS binary search for everything, even for packages inside the openSUSE distributions. Let's switch this to use repomd data from download.opensuse.org

    Ansible for add-on management by lmanfredi

    Description

    Machines can contains various combinations of add-ons and are often modified during the time.

    The list of repos can change so I would like to create an automation able to reset the status to a given state, based on metadata available for these machines

    Goals

    Create an Ansible automation able to take care of add-on (repo list) configuration using metadata as reference

    Resources

    Results

    Created WIP project Ansible-add-on-openSUSE

    "autoremove" functionality for zypper by e_bischoff

    The purpose would be to have the equivalent of Ubuntu's "apt-get autoremove" functionality.

    When you install package P, it might draw in dependancies D1, D2, ... Dn automatically.

    When you later uninstall P, the dependancies D1, D2, ..., Dn might remain on your system.

    If you keep installing and uninstalling packages, after a while your system remains cluttered with things you don't need.

    The idea would be to mark all dependancies that were installed but not explicitely requested as "installed automatically". Then a command like "zypper autoremove" could remove them at once if they are not needed anymore.

    Approaches seen online

    After scouring forms for solutions these are some alias's used to replicate the functionality

    1. zypper packages --unneeded | awk -F'|' 'NR==0 || NR==1 || NR==2 || NR==3 || NR==4 {next} {print $3}' | grep -v Name | sudo xargs zypper remove --clean-deps

    2. This one is a script and has bashisms bash mapfile -t unneeded < <(zypper --quiet pa --unneeded | awk '$1 == "i" { print $5, "-", $7 }') (( ${#unneeded@]} )) && sudo zypper --quiet rm --clean-deps --details "${unneeded@]}"

    3. sudo zypper rm $(zypper pa --unneeded | awk '/i / {print $3}' FS='|' | uniq | tr -d ' ')

    Based on testing zypper packages --orphaned provides packages that are not in any repo, even if a user has explicitly installed them, so --orphaned may not be the way to go, instead focusing on --unneeded