Currently the upstream LTP is executed by a hacked up and old runltp script that executes even worse and fairly old mess called ltp-pan which in turn actually executes the test cases. This whole thing is a unmaintainable mess that should have been replaced with something simpler a long time ago. It should also have a few more features that has been requested in the meantime and not implemented since nobody wants to touch the code. For instance executing the test cases on a different machine via ssh and writing the results locally. Another feature I've been thinking about for quite some time is a parallel test execution, since most of the test cases in fact could be executed in parallel which could easily speed up the test run twice. There are other tests that cannot, mostly stress tests, but also test cases that modify global system state, i.e. system time, make use of sysv IPC, use loop devices, etc. These kind of tests should be annotated somehow so that we do not end up with a test cases competing for a global resources in a parallel test run.

Looking for hackers with the skills:

hw15challenge

This project is part of:

Hack Week 15 Hack Week 17

Activity

  • about 6 years ago: pvorel liked this project.
  • over 6 years ago: pgonin liked this project.
  • over 6 years ago: zzhou liked this project.
  • over 6 years ago: hurhaj liked this project.
  • over 7 years ago: metan added keyword "hw15challenge" to this project.
  • over 7 years ago: dsterba liked this project.
  • over 7 years ago: sebchlad liked this project.
  • over 7 years ago: pluskalm liked this project.
  • over 7 years ago: mwilck liked this project.
  • over 7 years ago: metan started this project.
  • over 7 years ago: metan originated this project.

  • Comments

    • metan
      over 7 years ago by metan | Reply

      The proof of concept code is at:

      https://github.com/metan-ucw/ltp

      After installation you can try it by running the runltp-ng script, either with -o option that runs open posix testsuite or with a path to a syscall runtest file as a last argument.

      What I managed to finish:

      • Working proof of concent LTP testrunner that more or less can do everything that the old one could have done and more:
        • testcases are killed on timeout and the test run carries on!
        • the test runner can produce nice clickable (javascript) html page with results
        • the test runtime is measured and included in results

      What is still on TODO:

      • Parallel test execution (but the infrastructure is there!)
      • Remove test execution (should be easy, since we have clean enough codebase now)

    • metan
      over 7 years ago by metan | Reply

      Example test runner output:

      (it's still missing information about hardware/OS release/etc. which will be done later)

      http://metan.ucw.cz/outgoing/ltp-openposix.html

      http://metan.ucw.cz/outgoing/ltp-syscalls.html

    • metan
      over 6 years ago by metan | Reply

      The requirements have changed a bit meanwhile, so basically it's needed to split the testrunner and tested systems into two different machines to avoid incomplete results on kernel crashes.

      I will try to prototype proof of concept that will take a path to a qcow image of an virtual machine, execute testcases there and produce results.

    • metan
      about 6 years ago by metan | Reply

      The prototype has been finished, it's a bit messy perl code but the basic functionality is there.

      Here you can see output put from the testruns with a kernel trace in the middle followed by a reboot of the VM:

      http://metan.ucw.cz/outgoing/log.html

      The work is for now in my home repo in GitHub at:

      https://github.com/metan-ucw/ltp/tree/master/tools/runltp-ng

    Similar Projects

    This project is one of its kind!