sebchlad

Right now internal SLE development is still organised & structured around the concept of 'Milestones'. Schedules are defined, deadlines are set, and off we go making Alpha 1, 2, 3, Betas 1, 2, 3, RC's, and so on. Meanwhile, QA has evolved, and with openQA and other automated tooling we are increasingly testing SLE in a more agile, rolling model, testing every single build as soon as it's produced by OBS, and just paying extra attention to the Milestones with additional manual testing.

Currently the upstream LTP is executed by a hacked up and old runltp script that executes even worse and fairly old mess called ltp-pan which in turn actually executes the test cases. This whole thing is a unmaintainable mess that should have been replaced with something simpler a long time ago. It should also have a few more features that has been requested in the meantime and not implemented since nobody wants to touch the code. For instance executing the test cases on a different machine via ssh and writing the results locally. Another feature I've been thinking about for quite some time is a parallel test execution, since most of the test cases in fact could be executed in parallel which could easily speed up the test run twice. There are other tests that cannot, mostly stress tests, but also test cases that modify global system state, i.e. system time, make use of sysv IPC, use loop devices, etc. These kind of tests should be annotated somehow so that we do not end up with a test cases competing for a global resources in a parallel test run.