Project Description
zswap [1] is a linux kernel component that provides in-memory compression for swap pages. It already provides a limited form of deduplication: if a page is filled with the same value (e.g. all-zeroes) then only that value is kept instead of compressing the contents.
This project proposes to enhance zswap with a more general and powerful dedup capability: pages that contain the same content (as identified by a checksum or hash value across the entire page contents) will be stored only once in the compressed pool.
This would be somewhat similar in spirit to what ksm [2] does, that is consolidation of identical pages of anonymous memory. There are some differences though, primarily ksm requires userspace to explicitly opt-in (via madvise/MADV_MERGEABLE) and operates on active pages that have not been selected for eviction from memory. On the other hand zswap operates transparently from the applications (it applies to all pages of anonymous memory that are on their way to swap).
Goal for this Hackweek
Have a first working POC.
Resources
- [1] https://www.kernel.org/doc/html/latest/admin-guide/mm/zswap.html
- [2] https://www.kernel.org/doc/html/latest/admin-guide/mm/ksm.html
Looking for hackers with the skills:
This project is part of:
Hack Week 21
Activity
Comments
Be the first to comment!
Similar Projects
RISC-V emulator in GLSL capable of running Linux by favogt
Description
There are already numerous ways to run Linux and some programs through emulation in a web browser (e.g. x86 and riscv64 on https://bellard.org/jslinux/), but none use WebGL/WebGPU to run the emulation on the GPU.
I already made a PoC of an AArch64 (64-bit Arm) emulator in OpenCL which is unfortunately hindered by a multitude of OpenCL compiler bugs on all platforms (Intel with beignet or the new compute runtime and AMD with Mesa Clover and rusticl). With more widespread and thus less broken GLSL vs. OpenCL and the less complex implementation requirements for RV32 (especially 32bit integers instead of 64bit), that should not be a major problem anymore.
Goals
Write an RISC-V system emulator in GLSL that is capable of booting Linux and run some userspace programs interactively. Ideally it is small enough to work on online test platforms like Shaderoo with a custom texture that contains bootstrap code, kernel and initrd.
Minimum:
riscv32 without FPU (RV32 IMA) and MMU (µClinux), running Linux in M-mode and userspace in U-mode.
Stretch goals:
FPU support, S-Mode support with MMU, SMP. Custom web frontend with more possibilities for I/O (disk image, network?).
Resources
RISC-V ISA Specifications
Shaderoo
OpenGL 4.5 Quick Reference Card
Result as of Hackweek 2024
WebGL turned out to be insufficient, it only supports OpenGL ES 3.0 but imageLoad/imageStore needs ES 3.1. So we switched directions and had to write a native C++ host for the shaders.
As of Hackweek Friday, the kernel attempts to boot and outputs messages, but panics due to missing memory regions.
Since then, some bugs were fixed and enough hardware emulation implemented, so that now Linux boots with framebuffer support and it's possible to log in and run programs!
The repo with a demo video is available at https://github.com/Vogtinator/risky-v
Hacking on sched_ext by flonnegren
Description
Sched_ext upstream has some interesting issues open for grabs:
Goals
Send patches to sched_ext upstream
Also set up perfetto to trace some of the example schedulers.
Resources
https://github.com/sched-ext/scx
Improve UML page fault handler by ptesarik
Description
Improve UML handling of segmentation faults in kernel mode. Although such page faults are generally caused by a kernel bug, it is annoying if they cause an infinite loop, or panic the kernel. More importantly, a robust implementation allows to write KUnit tests for various guard pages, preventing potential kernel self-protection regressions.
Goals
Convert the UML page fault handler to use oops_* helpers, go through a few review rounds and finally get my patch series merged in 6.14.
Resources
Wrong initial attempt: https://lore.kernel.org/lkml/20231215121431.680-1-petrtesarik@huaweicloud.com/T/
Modularization and Modernization of cifs.ko for Enhanced SMB Protocol Support by hcarvalho
Creator:
Enzo Matsumiya ematsumiya@suse.de @ SUSE Samba team
Members:
Henrique Carvalho henrique.carvalho@suse.com @ SUSE Samba team
Description
Split cifs.ko in 2 separate modules; one for SMB 1.0 and 2.0.x, and another for SMB 2.1, 3.0, and 3.1.1.
Goals
Primary
Start phasing out/deprecation of older SMB versions
Secondary
- Clean up of the code (with focus on the newer versions)
- Update cifs-utils
- Update documentation
- Improve backport workflow (see below)
Technical details
Ideas for the implementation.
- fs/smb/client/{old,new}.c to generate the respective modules
- Maybe don't create separate folders? (re-evaluate as things progresses!)
- Remove server->{ops,vals} if possible
- Clean up fs_context.* -- merge duplicate options into one, handle them in userspace utils
- Reduce code in smb2pdu.c -- tons of functions with very similar init/setup -> send/recv -> handle/free flow
- Restructure multichannel
- Treat initial connection as "channel 0" regardless of multichannel enabled/negotiated status, proceed with extra channels accordingly
- Extra channel just point to "channel 0" as the primary server, no need to allocate an extra TCPServerInfo for each one
- Authentication mechanisms
- Modernize algorithms (references: himmelblau, IAKERB/Local KDC, SCRAM, oauth2 (Azure), etc.
early stage kdump support by mbrugger
Project Description
When we experience a early boot crash, we are not able to analyze the kernel dump, as user-space wasn't able to load the crash system. The idea is to make the crash system compiled into the host kernel (think of initramfs) so that we can create a kernel dump really early in the boot process.
Goal for the Hackweeks
- Investigate if this is possible and the implications it would have (done in HW21)
- Hack up a PoC (done in HW22 and HW23)
- Prepare RFC series (giving it's only one week, we are entering wishful thinking territory here).
update HW23
- I was able to include the crash kernel into the kernel Image.
- I'll need to find a way to load that from
init/main.c:start_kernel()
probably afterkcsan_init()
- I workaround for a smoke test was to hack
kexec_file_load()
systemcall which has two problems:- My initramfs in the porduction kernel does not have a new enough kexec version, that's not a blocker but where the week ended
- As the crash kernel is part of init.data it will be already stale once I can call
kexec_file_load()
from user-space.
The solution is probably to rewrite the POC so that the invocation can be done from init.text (that's my theory) but I'm not sure if I can reuse the kexec infrastructure in the kernel from there, which I rely on heavily.
update HW24
- Day1
- rebased on v6.12 with no problems others then me breaking the config
- setting up a new compilation and qemu/virtme env
- getting desperate as nothing works that used to work
- Day 2
- getting to call the invocation of loading the early kernel from
__init
afterkcsan_init()
- getting to call the invocation of loading the early kernel from
Day 3
- fix problem of memdup not being able to alloc so much memory... use 64K page sizes for now
- code refactoring
- I'm now able to load the crash kernel
- When using virtme I can boot into the crash kernel, also it doesn't boot completely (major milestone!), crash in
elfcorehdr_read_notes()
Day 4
- crash systems crashes (no pun intended) in
copy_old_mempage()
link; will need to understand elfcorehdr... - call path
vmcore_init() -> parse_crash_elf_headers() -> elfcorehdr_read() -> read_from_oldmem() -> copy_oldmem_page() -> copy_to_iter()
- crash systems crashes (no pun intended) in
Day 5
- hacking
arch/arm64/kernel/crash_dump.c:copy_old_mempage()
to see if crash system really starts. It does. - fun fact: retested with more reserved memory and with UEFI FW, host kernel crashes in init but directly starts the crash kernel, so it works (somehow) \o/
- hacking
TODOs
- fix elfcorehdr so that we actually can make use of all this...
- test where in the boot
__init()
chain we can/should callkexec_early_dump()