Re: ANN: leancheck-v0.7.4 with providers for Tasty, Hspec and test-framework

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: ANN: leancheck-v0.7.4 with providers for Tasty, Hspec and test-framework

Rudy Matela
Hello Haskell-Café,

A new version of LeanCheck is out: v0.7.4.  LeanCheck is a property
testing library (like QuickCheck) that tests values enumeratively rather
than at random (unlike QuickCheck).

_Whats new?_  I have created providers to make it easy to incorporate
LeanCheck properties in Tasty, Hspec and test-framework test suites.  This
is optional and provided in separate packages: LeanCheck can still be used
alone.

LeanCheck and its framework providers are available on Hackage.
You can install them with:

$ cabal install leancheck
$ cabal install tasty-leancheck
$ cabal install hspec-leancheck
$ cabal install test-framework-leancheck

Check out the README files for examples of use:

* https://github.com/rudymatela/leancheck
* https://github.com/rudymatela/tasty-leancheck
* https://github.com/rudymatela/hspec-leancheck
* https://github.com/rudymatela/test-framework-leancheck


Here's a sneak peek of LeanCheck+Tasty:


## Test program

    import Test.Tasty
    import Test.Tasty.LeanCheck as LC
    import Data.List

    main :: IO ()
    main = defaultMain tests

    tests :: TestTree
    tests = testGroup "Test properties checked by LeanCheck"
      [ LC.testProperty "sort == sort . reverse" $
          \list -> sort (list :: [Int]) == sort (reverse list)
      , LC.testProperty "Fermat's little theorem" $
          \x -> ((x :: Integer)^7 - x) `mod` 7 == 0
      -- the following property do not hold
      , LC.testProperty "Fermat's last theorem" $
          \x y z n ->
            (n :: Integer) >= 3 LC.==> x^n + y^n /= (z^n :: Integer)
      ]


## Output for the test program

    $ ./test
    Test properties checked by LeanCheck
      sort == sort . reverse:  OK
        +++ OK, passed 200 tests.
      Fermat's little theorem: OK
        +++ OK, passed 200 tests.
      Fermat's last theorem:   FAIL
        *** Failed! Falsifiable (after 71 tests):
        0 0 0 3

    1 out of 3 tests failed (0.00s)

You can't see it here, but Tasty's output is highlighted in colours:
"FAIL" appears in red so you can quickly see which properties failed.


Best Regards,
Rudy
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: ANN: leancheck-v0.7.4 with providers for Tasty, Hspec and test-framework

Johannes Waldmann-2
Hi.

I do prefer the enumerative approach for test case generation
over the randomized one (quickcheck) because I have use cases
(testing student code) where I want to avoid IO,
and I just want a pure list of (small) counterexamples.

Among the enumerative testing libraries,
the "competition" is between leancheck and smallcheck.
For, me the important differences are

Enumeration of algebraic data types (ADTs), via built-in
serial combinators, e.g.,  cons0 Leaf \/ cons2 Branch

* leancheck enumerate by size (number of nodes in tree)
* smallcheck enumerates by depth (longest path in tree)

I (much!) prefer "by size"
because this delays combinatorial explosion somewhat.

Automated Listable/Serial instances for ADTs

* leancheck uses template-haskell
* smallcheck uses DeriveGeneric

Here I would actually prefer DeriveGeneric.
But this certainly could be added to leancheck?

Oh, and leancheck does not use fancy typeclassery
(smallcheck needs Multiparamtypeclasses and Monads
just to declare a Serial instance)
and it is also easier to get at the list of counterexamples.

- J.W.
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: ANN: leancheck-v0.7.4 with providers for Tasty, Hspec and test-framework

Rudy Matela
Hi,

I'm glad you liked LeanCheck.  :-)

On Mon, 10 Sep 2018 15:09:43 +0200, Johannes Waldmann wrote:

> Among the enumerative testing libraries,
> the "competition" is between leancheck and smallcheck.
> For, me the important differences are
>
> Enumeration of algebraic data types (ADTs), via built-in
> serial combinators, e.g.,  cons0 Leaf \/ cons2 Branch
>
> * leancheck enumerate by size (number of nodes in tree)
> * smallcheck enumerates by depth (longest path in tree)
>
> I (much!) prefer "by size"
> because this delays combinatorial explosion somewhat.

Yes, size bounded enumeration is more tractable.  The combinatorial
explosion is still there, but a lot of the times much later in the
enumeration.

LeanCheck also avoids repeating tests.  SmallCheck may/will repeat tests
when testing successive "depths".

> Automated Listable/Serial instances for ADTs
>
> * leancheck uses template-haskell
> * smallcheck uses DeriveGeneric
>
> Here I would actually prefer DeriveGeneric.

Might I ask why the preference?

> But this certainly could be added to leancheck?

Yes it could.  I guess LeanCheck could provide a genericTiers /
defaultTiers function (of type something like `Generic a => [[a]]`) on an
optional module so that one would write:

        import Test.LeanCheck.Generic

        instance Listable MyType where
          tiers = genericTiers

Patches or GitHub pull requests are always welcome.  :-)  -- I may add
this myself if I ever get the time.

Another addition could be defining a "default" inside the Listable
typeclass itself so that one could write `data MyType = ... deriving
Listable`.  But I'm not very keen on this second option, as it would
require an extension on LeanCheck's core which is compliant with
Haskell98+Namespaces.  It would also hinder the beginner-friendliness of
reading that module.  (Or maybe there is a way to define "orphan" default
declarations for typeclasses in separate modules?)

> Oh, and leancheck does not use fancy typeclassery
> (smallcheck needs Multiparamtypeclasses and Monads
> just to declare a Serial instance)
> and it is also easier to get at the list of counterexamples.

This was intentional, I'm a fan of simple types and interfaces.  I'm glad
you liked this.

- Rudy
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.