Description
Why not... ALM and a new test framework?
Goals
- ✅ Development environment set up (Rust + Python)
- ✅ Basic eBPF tracing capturing syscall activity
- ✅ Local LLM generating test scenarios from syscall documentation
- ✅ 5-10 executable test cases for a single syscall family
- ✅ Simple pass/fail validation mechanism
- ✅ End-to-end demo working on local machine
Resources
https://github.com/d3flex/Delphos/
No Hackers yet
Looking for hackers with the skills:
Nothing? Add some keywords!
This project is part of:
Hack Week 25
Comments
-
10 days ago by ybonatakis | Reply
Delphos - Day1
All the objectives are reached for the first day without any serious issues. I managed to create a very basic structure for the project, leaving out for now any design pattern logic.
Then I had to prepare some dependencies. I declared before, I am planning to use Rust and Python. For the later, the only dependency i had was
uv.bash sudo zypper in python3-uvThat is my first ever interaction with Rust. From the documentation I found that I need to install
rustup. I want to avoid the install script, so I search zypper packages.```bash ❯ sudo zypper in rustup Refreshing service 'openSUSE'. Loading repository data... Reading installed packages... Resolving package dependencies...
Problem: 1: the to be installed rustup-1.28.2~0-2.1.x8664 conflicts with 'rust+rustc' provided by the installed rust1.91-1.91.0-2.1.x8664 Solution 1: Following actions will be done: deinstallation of rust1.91-1.91.0-2.1.x8664 deinstallation of cargo1.91-1.91.0-2.1.x8664 deinstallation of cargo-1.91.0-1.1.x8664 deinstallation of rust-1.91.0-1.1.x8664 Solution 2: do not install rustup-1.28.2~0-2.1.x86_64
Choose from above solutions by number or cancel 1/2/c/d/?: ```
I found that I had already install rust as package and that was just enought to run a hello-world.
The last requirement was ollama. From the web site I found the latest llama and I run it with
ollama run llama3.2:3bThe following day was pretty much reading the first chapters from the Rust Documentation.
Delphos - day2
Day 2 was an interesting one, looking into bpftrace documentation and trying things in terminal. I cant really say I read it extensively but I think I got a basic idea of what I need and how that would look like. I ended up with a small file to run syscall cmds.
The plan was to implement the ollama interface in python. that would generate some test scenario and then I would have to work on the Rust code to run it. However I decided to make some basic refactoring in the structure and the current code, because the end goal is to provide a good design pattern for the whole idea.
With this I started added individual Classes representing the various
actors. I am still not satisfied or done but I had to move on in order to have something actually running soon. Some tech dept are expected and are invetable.The last (hard) part was the code which would take the test scenario and actually run them and give the results. Despite the study of the Rust the previous day, when I came to write the real code I struggled a lot. So I tried line by line, and back and forth with AI assistant, in order to understand and write the minimun workable code.
At least and at last, I saw it running.
Delphos - day3 and day4
I log those days together. There were no much happening in day3, the little time I found to look into the project was some minor things and mainly read Rust documentation.
However as i write these lines, the main goal is achieved. The was a massive refactoring, as I couldnt get out of my head that I wanted to have at least a solid design. So I tried to balannce the plan with some additional desires, keeping in mind that I need a MVP within a 5 days period. So there are compromises. Having said that, I consider the concern in the design an important factor for the success of any project, so i am kinda happy with the sacrisfy.
Now the Delphos can use any source a user wants to add extending and/or implemented the generator.sources.base class, following the Strategy pattern. As for now, there is only an abstraction on the Documents. The next design decision was on the ContextBuilder. it is implement the Facade Pattern, in order unify and provide an interface to access the sources.
What works: - command line with parameters to interact with the main components - A llm interface. at the moment uses a hardcoded ollama instance but can take any model. - a simplified generator for any context source which stores the test scanarios locally. - even simplier orchestrator. (the implementation is hardcoded)
What does not work: - orchestrator doesnt really read the test_scenario description - proper test environment. the tests run against the local machine - propper logging not inmplemented - only manpages as context source - Unit testing
Delphos - day5
TBD
Similar Projects
This project is one of its kind!