Currently the upstream LTP is executed by a hacked up and old runltp script that executes even worse and fairly old mess called ltp-pan which in turn actually executes the test cases. This whole thing is a unmaintainable mess that should have been replaced with something simpler a long time ago. It should also have a few more features that has been requested in the meantime and not implemented since nobody wants to touch the code. For instance executing the test cases on a different machine via ssh and writing the results locally. Another feature I've been thinking about for quite some time is a parallel test execution, since most of the test cases in fact could be executed in parallel which could easily speed up the test run twice. There are other tests that cannot, mostly stress tests, but also test cases that modify global system state, i.e. system time, make use of sysv IPC, use loop devices, etc. These kind of tests should be annotated somehow so that we do not end up with a test cases competing for a global resources in a parallel test run.
Looking for hackers with the skills:
This project is part of:
Hack Week 15 Hack Week 17
Activity
Comments
-
about 7 years ago by metan | Reply
The proof of concept code is at:
https://github.com/metan-ucw/ltp
After installation you can try it by running the runltp-ng script, either with -o option that runs open posix testsuite or with a path to a syscall runtest file as a last argument.
What I managed to finish:
- Working proof of concent LTP testrunner that more or less can do everything
that the old one could have done and more:
- testcases are killed on timeout and the test run carries on!
- the test runner can produce nice clickable (javascript) html page with results
- the test runtime is measured and included in results
What is still on TODO:
- Parallel test execution (but the infrastructure is there!)
- Remove test execution (should be easy, since we have clean enough codebase now)
- Working proof of concent LTP testrunner that more or less can do everything
that the old one could have done and more:
-
about 7 years ago by metan | Reply
Example test runner output:
(it's still missing information about hardware/OS release/etc. which will be done later)
-
almost 6 years ago by metan | Reply
The requirements have changed a bit meanwhile, so basically it's needed to split the testrunner and tested systems into two different machines to avoid incomplete results on kernel crashes.
I will try to prototype proof of concept that will take a path to a qcow image of an virtual machine, execute testcases there and produce results.
-
almost 6 years ago by metan | Reply
The prototype has been finished, it's a bit messy perl code but the basic functionality is there.
Here you can see output put from the testruns with a kernel trace in the middle followed by a reboot of the VM:
http://metan.ucw.cz/outgoing/log.html
The work is for now in my home repo in GitHub at:
https://github.com/metan-ucw/ltp/tree/master/tools/runltp-ng
Similar Projects
This project is one of its kind!