I have previously talked about using easyb for unit and/or acceptance testing.
When one talks about testing, one should always also talk about “code coverage.” This is true regardless whether one is talking about Pascal or Prolog, C or Groovy.
One of the most effective tools for coverage testing for Groovy is cobertura
Since the festive season is nearly upon us, I have created a small Groovy class that generates the old, Catechism Song “The Twelve Days of Christmas”:
public class Christmas {
static ordinal(d) {
def s
switch (d) {
case 1: s = "1st"; break
case 2: s = "2nd"; break
case 3: s = "3rd"; break
default: s = (d + "th"); break
}
s
}
static line(l) {
"t$ln"
}
static verse(day) {
def s = new StringBuilder("On the ${ordinal(day)} day of Christmas my true love gave to me:")
s << 'n'
if (day >= 12)
s << line("twelve drummers drumming,")
if (day >= 11)
s << line("eleven pipers piping,")
if (day >= 10)
s << line("ten lords a-leaping,")
if (day >= 9)
s << line("nine ladies dancing,")
if (day >= 8)
s << line("eight maids a-milking,")
if (day >= 7)
s << line("seven swans a-swimming,")
if (day >= 6)
s << line("six geese a-laying,")
if (day >= 5)
s << line("five gold rings,")
if (day >= 4)
s << line("four calling birds,")
if (day >= 3)
s << line("three french hens,")
if (day >= 2) {
s << line("two turtle doves")
s << line('and')
}
if (day >= 1)
s << line("a partridge in a pear tree.")
s
}
static void main(args) {
(0..11).each {day ->
println verse(day)
}
}
}
The algorithm may not be the best, but it works well for my purpose here so let’s see how Cobertura helps uncover the glaring off-by-one bug in this little work of art…
On the 0th day of Christmas my true love gave to me:
On the 1st day of Christmas my true love gave to me:
a partridge in a pear tree.On the 2nd day of Christmas my true love gave to me:
two turtle doves
and
a partridge in a pear tree.…elided…
On the 11th day of Christmas my true love gave to me:
eleven pipers piping,
ten lords a-leaping,
nine ladies dancing,
eight maids a-milking,
seven swans a-swimming,
six geese a-laying,
five gold rings,
four calling birds,
three french hens,
two turtle doves
and
a partridge in a pear tree.
Using cobertura requires a three-step process. in the first step, Cobertura instruments java class files so that usage counts are maintained. In the second step, the system under test’s runtime classpath is changed to ensure that these instrumented classes are used in place of the originals. The third step takes place after execution when Cobertura analyses the data and generates a report.
Take a look at the following gant script to see how all that is done:
DEVTOOLS = 'c:/DEVTOOLS'
dirSource = 'src'
dirBuild = 'out'
dirCoberturaHome = "${DEVTOOLS}/cobertura-1.8"
dirCoberturaClasses = dirBuild + '/coberturaClasses'
dirCoberturaReports = 'coberturaReports'
fileCoberturaData = "cobertura.ser"
includeTargets << gant.targets.Clean
cleanPattern << '**/*~'
cleanDirectory << [ dirCoberturaReports ]
ant.path(id: 'pathCobertura') {
fileset(dir: dirCoberturaHome, includes: 'lib/**/*.jar, cobertura.jar')
}
ant.taskdef(resource: 'tasks.properties', classpathref: 'pathCobertura')
target(coberturaInstrumentation: 'Run Cobertura instrumentation') {
ant.'cobertura-instrument'(todir: dirCoberturaClasses) {
fileset(dir: dirBuild, includes: '*.class')
}
}
target(coberturaReports: 'Run Cobertura reporting') {
ant.'cobertura-report'(format: 'html', srcdir: dirSource,
destdir: dirCoberturaReports)
}
ant.taskdef(name: 'groovyc', classname: 'org.codehaus.groovy.ant.Groovyc')
target(compile: 'Compile source to build directory') {
groovyc(srcdir: dirSource, destdir: dirBuild) {
javac(debug: 'on', debuglevel: 'lines,vars,source')
}
}
target(christmas: 'Run Christmas application') {
java(classname: 'Christmas', fork:true) {
classpath() {
pathelement(path: "C:/DEVTOOLS/groovy-1.6-beta-2/embeddable/groovy-all-1.6-beta-2.jar")
pathelement(path: dirCoberturaClasses)
path(refid: 'pathCobertura')
}
}
}
target(init: 'Initialise the build, given a clean start') {
depends(clean)
ant.mkdir(dir: dirCoberturaReports)
if (new File(fileCoberturaData).exists()) ant.delete(file: fileCoberturaData)
}
target(defaultTarget: 'Do Everything') {
depends(init)
compile()
coberturaInstrumentation()
christmas()
coberturaReports()
}
setDefaultTarget(defaultTarget)
For this simple class, the coverage report points us straight to a problem: The twelfth verse is never requested.
the cause is simple,
static void main(args) {
(0..11).each {day ->
println verse(day)
}
}
instead of:
static void main(args) {
(1..12).each {day ->
println verse(day)
}
Well OK. Not so impressive it seems! After all in this situation we have a straightforward class and straightforward test. A simple code-inspection would probably have surfaced the bug, but you never know: this class of error is depressingly prevalent in code.
Code coverage is valuable when testing is more complex and the System Under Test is much larger; it comes into is own when testing is the responsibility of all the members of a team (something that is a central tenet of XP, after all).
Getting Cobertura up and running is easy. Getting it integrated into a Continuous Integration system like hudson is also easy.
(By the way, if you were wondering: the price tag of the shopping list laid out in the classic holiday song “The Twelve Days of Christmas” is sharply higher this year.)
