SUSE Hack Week: generic zswap dedup

Project Description

zswap [1] is a linux kernel component that provides in-memory compression for swap pages. It already provides a limited form of deduplication: if a page is filled with the same value (e.g. all-zeroes) then only that value is kept instead of compressing the contents.

This project proposes to enhance zswap with a more general and powerful dedup capability: pages that contain the same content (as identified by a checksum or hash value across the entire page contents) will be stored only once in the compressed pool.

This would be somewhat similar in spirit to what ksm [2] does, that is consolidation of identical pages of anonymous memory. There are some differences though, primarily ksm requires userspace to explicitly opt-in (via madvise/MADV_MERGEABLE) and operates on active pages that have not been selected for eviction from memory. On the other hand zswap operates transparently from the applications (it applies to all pages of anonymous memory that are on their way to swap).

Goal for this Hackweek

Have a first working POC.

Resources

[1] https://www.kernel.org/doc/html/latest/admin-guide/mm/zswap.html
[2] https://www.kernel.org/doc/html/latest/admin-guide/mm/ksm.html

Join this project Leave this project

Looking for hackers with the skills:

kernel

This project is part of:

Hack Week 21

Activity

almost 3 years ago: ailiopoulos liked this project.

almost 3 years ago: dmdiss liked this project.

almost 3 years ago: shunghsiyu liked this project.

almost 3 years ago: ailiopoulos added keyword "kernel" to this project.

almost 3 years ago: mbrugger liked this project.

almost 3 years ago: ailiopoulos started this project.

almost 3 years ago: ailiopoulos originated this project.

Comments

Be the first to comment!

Similar Projects

kernel

Create DRM drivers for VESA and EFI framebuffers by tdz

Description

We already have simpledrm for firmware framebuffers. But the driver is originally for ARM boards, not PCs. It is already overloaded with code to support both use cases. At the same time it is missing possible features for VESA and EFI, such as palette modes or EDID support. We should have DRM drivers for VESA and EFI interfaces. The infrastructure exists already and initial drivers can be forked from simpledrm.

Goals

Initially, a bare driver for VESA or EFI should be created. It can take functionality from simpledrm.
Then we can begin to add additional features. The boot loader can provide EDID data. With VGA hardware, VESA can support paletted modes or color management. Example code exists in vesafb.

Linux on Cavium CN23XX cards by tsbogend

Before Cavium switched to ARM64 CPUs they developed quite powerful MIPS based SOCs. The current upstream Linux kernel already supports some Octeon SOCs, but not the latest versions. Goal of this Hack Week project is to use the latest Cavium SDK to update the Linux kernel code to let it running on CN23XX network cards.

early stage kdump support by mbrugger

Project Description

When we experience a early boot crash, we are not able to analyze the kernel dump, as user-space wasn't able to load the crash system. The idea is to make the crash system compiled into the host kernel (think of initramfs) so that we can create a kernel dump really early in the boot process.

Goal for the Hackweeks

Investigate if this is possible and the implications it would have (done in HW21)
Hack up a PoC (done in HW22 and HW23)
Prepare RFC series (giving it's only one week, we are entering wishful thinking territory here).

update HW23

I was able to include the crash kernel into the kernel Image.
I'll need to find a way to load that from init/main.c:start_kernel() probably after kcsan_init()
I workaround for a smoke test was to hack kexec_file_load() systemcall which has two problems:
1. My initramfs in the porduction kernel does not have a new enough kexec version, that's not a blocker but where the week ended
2. As the crash kernel is part of init.data it will be already stale once I can call kexec_file_load() from user-space.

The solution is probably to rewrite the POC so that the invocation can be done from init.text (that's my theory) but I'm not sure if I can reuse the kexec infrastructure in the kernel from there, which I rely on heavily.

update HW24

Day1
- rebased on v6.12 with no problems others then me breaking the config
- setting up a new compilation and qemu/virtme env
- getting desperate as nothing works that used to work
Day 2
- getting to call the invocation of loading the early kernel from __init after kcsan_init()
Day 3
- fix problem of memdup not being able to alloc so much memory... use 64K page sizes for now
- code refactoring
- I'm now able to load the crash kernel
- When using virtme I can boot into the crash kernel, also it doesn't boot completely (major milestone!), crash in elfcorehdr_read_notes()
Day 4
- crash systems crashes (no pun intended) in copy_old_mempage() link; will need to understand elfcorehdr...
- call path vmcore_init() -> parse_crash_elf_headers() -> elfcorehdr_read() -> read_from_oldmem() -> copy_oldmem_page() -> copy_to_iter()
Day 5
- hacking arch/arm64/kernel/crash_dump.c:copy_old_mempage() to see if crash system really starts. It does.
- fun fact: retested with more reserved memory and with UEFI FW, host kernel crashes in init but directly starts the crash kernel, so it works (somehow) \o/
TODOs
- fix elfcorehdr so that we actually can make use of all this...
- test where in the boot __init() chain we can/should call kexec_early_dump()

Hacking on sched_ext by flonnegren

Description

Sched_ext upstream has some interesting issues open for grabs:

Goals

Send patches to sched_ext upstream

Also set up perfetto to trace some of the example schedulers.

Resources

https://github.com/sched-ext/scx

Improve various phones kernel mainline support (Qualcomm, Exynos, MediaTek) by pvorel

Result

In the end I concentrated again to msm8994: