Don't write tests! Generate them.
an invention by e_bischoff
Updated about 5 years ago. 7 hacker ♥️.

The title of this project is inspired from the must-see video

John Hughes - Testing the Hard Stuff and Staying Sane

I would like to play with QuickCheck to generate tests and possibly find bugs automatically.

I chose Wicked as the tested software, but it could be anything else.

My hope is to stay away from Haskell as much as possible as I have no real interest in learning Yet Another Programming Language. Ideally, I would reuse as much as possible from the existing Ruby testsuite for Wicked and stick to Ruby. There are projects like RubyCheck that are ports of QuickCheck to Ruby. But well, if I have to learn Haskell as part of this project, I would do it.

Feel welcome to join the project.

Looking for hackers with the skills:

wicked testing quickcheck

This project is part of:

Hack Week 13

Activity

  • about 7 years ago: thardeck disliked this project.
  • about 8 years ago: wawiv74 joined this project.
  • over 8 years ago: thardeck liked this project.
  • over 8 years ago: dgutu liked this project.
  • over 8 years ago: mvidner liked this project.
  • over 8 years ago: dwaas liked this project.
  • over 8 years ago: nadvornik liked this project.
  • over 8 years ago: oholecek liked this project.
  • over 8 years ago: dmaiocchi liked this project.
  • over 8 years ago: pgonin liked this project.
  • over 8 years ago: e_bischoff added keyword "wicked" to this project.
  • over 8 years ago: e_bischoff added keyword "testing" to this project.
  • over 8 years ago: e_bischoff added keyword "quickcheck" to this project.
  • over 8 years ago: e_bischoff started this project.
  • over 8 years ago: e_bischoff originated this project.


    • Comments

    • mvidner
      over 8 years ago by mvidner | Reply

      Regarding QuickCheck and Ruby, Balazs wrote something last year but I don't know how far he got: https://github.com/bkutil/exploring_quickcheck

    • e_bischoff
      over 8 years ago by e_bischoff | Reply

      Yes, apart from the choice of examples (Wicked in my case), it is more or less the same project.

      And as a matter of fact, I don't want to start with Wicked straight ahead. The plan was to implement in a first step more or less the same example as in the video : it would be an external C program that manages a queue (of integers, why not). I hope to do as many bugs as they do in the video :-) so RubyCheck has at least something to find.

      I would use Rantly too to generate the testcase integers. So yeah, a lot of similarities. Thanks for pointing that.

    • e_bischoff
      over 8 years ago by e_bischoff | Reply

      I just realized that Rantly implements QuickCheck too, it is not only for generating random data.

      So there are three out there: Rantly, RushCheck, RubyCheck . Not sure yet which one to take, that will be part of the exploration.

    • e_bischoff
      over 8 years ago by e_bischoff | Reply

      Nope, rantly does not implement QuickCheck, despite what it says. It is just a very convenient random input generator.

      Getting back to RubyCheck as initially planned.

    • e_bischoff
      over 8 years ago by e_bischoff | Reply

      RubyCheck too does no test failure reduction. Rantly at least has "shrinking", but it is shrinking of data types (strings, integers...), not a sequence of test cases... Do I have to map manually a sequence of test cases to such a data type?

      I also discovered Theft. Continuing exploration...

    • e_bischoff
      over 8 years ago by e_bischoff | Reply

      I ussceeded reducing (shrinking) with Rantly. Starting to save my experiments in my gitlab home project.

    • e_bischoff
      over 8 years ago by e_bischoff | Reply

      Hackweek finished. I did not make it in time testing Wicked with that way.

      I wrote a report with all my finding in the gitlab project:

      https://gitlab.suse.de/ebischoff/generate_tests/blob/master/hackweek.txt

    • e_bischoff
      over 8 years ago by e_bischoff | Reply

      I would need 3 days to finish the hackweek project and do its application to wicked.

    • e_bischoff
      almost 8 years ago by e_bischoff | Reply

      I'll consider the project as finished - Rantly has been improved, with my patches accepted upstream, and I played with the tool enough to know quite well his strengths and weaknesses.

      If I want to test wicked with it, it could be a separate hackweek project.

    Similar Projects