o continuous integration is an important part of your development process and you’re eager to include a code-coverage check as part of your automated builds, but which coverage-rate targets should your development team set? Seasoned supporters of code coverage suggest that you aim for 75 percent, 85 percent, or perhaps even 100 percent coverage rates.

After measuring the baseline coverage rates for a project at my organization, I realized our development team needed to set our sights much lower than those rate targets. I didn’t want to become the team’s Colonel Cathcart, the character from Josef Heller’s novel Catch 22 who arbitrarily raised his expectations as soon as it appeared his squadron would achieve the current ones:

Yossarian slumped with disappointment. “Then I really have to fly the fifty missions, don’t I?” he grieved.
“The fifty-five,” Doc Daneeka corrected him.
“What fifty-five?”
“The fifty-five missions the colonel now wants all of you to fly.”

?Excerpt from Catch 22 by Josef Heller

Rather than setting arbitrary targets, I chose the strategy of incremental improvement. To successfully execute this strategy, each build must have equal or better coverage than the previous successful build. By taking many small steps I hoped to achieve a giant leap in quality.

This article describes how you can implement the incremental improvement strategy in your code coverage using Cobertura and Apache Ant.

Unit Testing, Code Coverage, and Continuous Integration
Unit testing, code coverage, and continuous integration are all widely accepted best practices. In fact, most developers know they should unit test religiously. If you’re not already one of the converted, let me paraphrase Google Director of Research Peter Norvig:

If you think you don’t need to write unit tests for your code, write down all the reasons on a piece of paper. Study the paper carefully. Then throw away the paper and write tests anyway.

But who tests the tester? That is, how do you verify that you are writing enough tests? This is very valuable information because the code that isn’t being exercised by your tests is where you should focus your energy. One solution is using a code-coverage tool, which will tell you the percentage of code your tests are exercising, and then incorporating a coverage check in your normal integration process. If your coverage check fails, then your build should also fail.

For my incremental improvement strategy, I chose the code coverage tool Cobertura because of its simple, well-defined, four Ant-task interface. One of these tasks, cobertura-check, causes a build to fail when code doesn’t achieve the required coverage rate. For example, this Ant target will cause the build to fail if the coverage rate slips to less than 80 percent:

But instead of hard coding the line rate, you should use the result of an earlier build as the target rate for the current check. You can achieve this by chaining a couple of the Cobertura tasks with two core Ant tasks. And don’t worry about whether to measure line rate, branch rate, or some other coverage metric. Your goal is to achieve a marked improvement rather than setting an absolute target or worrying about the details of which coverage rate metric to choose.

(Read an introduction to using Cobertura here. If you use a tool that doesn’t have an equivalent to the cobertura-check task, check out the Ant fail task?but be prepared to write your own custom condition.)

Create a Coverage Report in XML
After you create the targets for instrumenting your code and running your tests, you can add incremental coverage rate checking to your build scripts. The first step is to use a cobertura-report task to create a coverage report in XML:

This is an example of the report (coverage.xml) a cobertura-report task produces:

                  ./src/java                  ...      

Make sure you save this file somewhere safe (perhaps even check it into your version management system) because you will need it later.

Extract the Coverage Rate from the Report
Initially, you might be tempted to use Ant’s XmlProperty task to extract the line rate directly to an Ant property. However, this approach is problematic for a couple of reasons:

1. A quirk of Cobertura is that the line-rate values in coverage.xml are decimal fractions, while the cobertura-check task requires integer percentages.
2. In real-life projects of significant size, coverage.xml is likely to be large in size, and any attempt to use the XmlProperty task will probably result in a Java OutOfMemoryError.

Instead, use the Ant XSLT task to extract only what you want:

This simple XSL template produces a properties file containing just the value that you need:

				total.line-rate=			

Note the method floor(@line-rate*100), which converts the line rate to a percentage and rounds it down to the nearest integer. The result is a coverage.properties file containing just one line:

total.line-rate=44

In contrast to coverage.xml, which you should save between builds, coverage.properties is just an intermediate stage that you can clean up after the build.

You can now use Ant’s property task to read the value from coverage.properties into an Ant property named total.line-rate:

Finally, you can replace the “80” in the original cobertura-check task with the newly set Ant property:

Putting It All Together
The final build.xml looks like this (the cobertura-instrument task, testing targets, and so on are omitted for clarity):

Note that a new coverage report is generated only when the coverage check has passed (that is, the coverage rate is as good as or better than the previous successful build).

Remember to bootstrap this approach by executing the coverage_report target before the very first execution of coverage_check. In practice, you should add another cobertura-report task to produce an HTML report too.

Tracking Rate of Improvement
A nice little add-on to the incremental improvement approach is tracking the rate of improvement by logging the coverage rate to a file. You accomplish this by using the Ant echo task:

                            ${date.time};total.line-rate;${total.line-rate}    

Measurable Results, Visible Improvement
A week after implementing this strategy in the project I mentioned in the introduction, we achieved a 30 percent coverage improvement (albeit from a very modest baseline). Most gratifyingly, developers who previously were reluctant to write tests are now taking pride in seeing the project’s coverage rates improve.

In the true democratic spirit of agile development, each and every team member can raise the target for the whole project by writing more tests?no arbitrary targets, no Colonel Cathcart.

And you don’t need to stop there. You can use the incremental improvement strategy for other code metrics too. Because most code-checking tools produce XML-formatted output, you can filter out the relevant metrics using an XSL template and use them as input values for checking in the following build.