Tools such as NCover and the Haskell Program Coverage tool, it can ensure our honesty when it comes to tests, and we get a glaring reminder when we don't. These tools, when combined with our traditional xUnit and property-based tests with saturation test generation can be a satisfying experience. We've now covered the creation and combination of traditional xUnit tests with property-based tests and how to leverage code coverage as a tool for refining. There is still more to be covered in this series which includes refactoring.
In the previous post, we talked about some of the basics of functional programming unit testing. That post mostly focused around HUnit, which is a traditional xUnit testing framework. This time, let's focus on type-based property testing, which is to create specs which assert logical properties about a function, and then to generate data to test in an attempt to falsify these assertions, through the use of a tool called QuickCheck. Much like the traditional xUnit frameworks, this tool helps us flush out the specifications of our software through the use of tests. Unlike the xUnit frameworks, however, this framework allows us to create generators to help flush out our behaviors and capture our edge cases as we look for ways to falsify our tests. These generators could use either random data or well structured data that you can craft. Let's dive a little deeper into what that means.
Let's focus on the tooling around testing with functional languages. What kind of options do we have? In the Haskell world just as the F# world, there are several tools at our disposal to do this. * HUnit A traditional xUnit testing framework for unit testing. Analogous to such frameworks as xUnit.net, NUnit and MbUnit in the .NET world. * QuickCheck A program in which the developer provides a specification of the program, in the form of properties which functions should satisfy, and then tests that the properties hold in a large number of randomly generated cases that QuickCheck provides. There are many variants of this tool for most functional languages including F# (FsCheck), Erlang, Scala, Java, Python, Standard ML and others. Today we're going to focus on HUnit as part of developing an API in Haskell.
In the functional programming world, we have two main ways of testing our code, either through the traditional xUnit tests or the more powerful QuickCheck property-based tests. Each of these are powerful in their own right, but made more powerful when combined into a single unit. When combined we have the power of integrating them into our CI process through some of the build/package tools in our tool belt.
ScalaCheck is a powerful tool for automatic unit testing of Scala and Java programs. It features automatic test case generation and minimization of failing test cases. ScalaCheck started out as a Scala port of the Haskell library QuickCheck, and has since evolved and been extended with features not found in Haskell QuickCheck.