The title of this project is inspired from the must-see video
John Hughes - Testing the Hard Stuff and Staying Sane
I would like to play with QuickCheck to generate tests and possibly find bugs automatically.
I chose Wicked as the tested software, but it could be anything else.
My hope is to stay away from Haskell as much as possible as I have no real interest in learning Yet Another Programming Language. Ideally, I would reuse as much as possible from the existing Ruby testsuite for Wicked and stick to Ruby. There are projects like RubyCheck that are ports of QuickCheck to Ruby. But well, if I have to learn Haskell as part of this project, I would do it.
Feel welcome to join the project.
Looking for hackers with the skills:
This project is part of:
Hack Week 13
Activity
Comments
-
almost 9 years ago by mvidner | Reply
Regarding QuickCheck and Ruby, Balazs wrote something last year but I don't know how far he got: https://github.com/bkutil/exploring_quickcheck
-
almost 9 years ago by e_bischoff | Reply
Yes, apart from the choice of examples (Wicked in my case), it is more or less the same project.
And as a matter of fact, I don't want to start with Wicked straight ahead. The plan was to implement in a first step more or less the same example as in the video : it would be an external C program that manages a queue (of integers, why not). I hope to do as many bugs as they do in the video :-) so RubyCheck has at least something to find.
I would use Rantly too to generate the testcase integers. So yeah, a lot of similarities. Thanks for pointing that.
-
almost 9 years ago by e_bischoff | Reply
I just realized that Rantly implements QuickCheck too, it is not only for generating random data.
So there are three out there: Rantly, RushCheck, RubyCheck . Not sure yet which one to take, that will be part of the exploration.
-
almost 9 years ago by e_bischoff | Reply
Nope, rantly does not implement QuickCheck, despite what it says. It is just a very convenient random input generator.
Getting back to RubyCheck as initially planned.
-
almost 9 years ago by e_bischoff | Reply
RubyCheck too does no test failure reduction. Rantly at least has "shrinking", but it is shrinking of data types (strings, integers...), not a sequence of test cases... Do I have to map manually a sequence of test cases to such a data type?
I also discovered Theft. Continuing exploration...
-
almost 9 years ago by e_bischoff | Reply
I ussceeded reducing (shrinking) with Rantly. Starting to save my experiments in my gitlab home project.
-
almost 9 years ago by e_bischoff | Reply
Hackweek finished. I did not make it in time testing Wicked with that way.
I wrote a report with all my finding in the gitlab project:
https://gitlab.suse.de/ebischoff/generate_tests/blob/master/hackweek.txt
-
almost 9 years ago by e_bischoff | Reply
I would need 3 days to finish the hackweek project and do its application to wicked.
-
over 8 years ago by e_bischoff | Reply
I'll consider the project as finished - Rantly has been improved, with my patches accepted upstream, and I played with the tool enough to know quite well his strengths and weaknesses.
If I want to test wicked with it, it could be a separate hackweek project.
Similar Projects
This project is one of its kind!