Two Critical Tips for Unit Testing in Java : Page 3

Some code is non-deterministic. In other words, the detailed results of a method are influenced by more than just the code in the method. Probably the simplest example of this is the return value of the System.currentTimeMillis() method. The exact results depend not on the code but on the underlying hardware. It effectively returns a value based on the system clock—a value that probably will be different each time the code is run.

Another example could be testing the time it takes a message to travel from a Web server to a browser over the Internet. This concept has too many variables to even create a predictable model. The Internet at a fine level of granularity is non-deterministic—even chaotic.

However, just because something isn't predictable at a fine level of granularity doesn't mean that its overall behavior isn't predictable. If you aggregate the fine grain results into a larger figure, say an arithmetic mean, then you could make some useful predictions about the expected values.

For instance, you could have a requirement that 90 percent of all Web-based transactions should complete within 1/100th of a second. Your technique here would be simply to run the test code many times (in a loop) and compare the transaction times with one second, keeping track of the number of passes and fails. If at the end of the test less than 90 percent of the transactions fail, then the test fails too.

A test uses the following code to confirm that the performance of a transaction is within acceptable bounds:

private int doPerformanceTest(int numberOfRuns, int 
   requiredTimeInMilliseconds) {
  int passed = 0;
  Fragment fragment = new Fragment();

  for(int i = 0; i < numberOfRuns; i++) {
    long startTime = System.currentTimeMillis();
    fragment.doTransaction();
    long endTime = System.currentTimeMillis();
    long runTime = endTime - startTime;
    if(runTime < requiredTimeInMilliseconds) passed++;
  }
  return passed;
}

The test that uses this method is responsible for asserting the performance criteria that should be met:

public void testPerformanceStatistics () {
  double percentageRequired = 90.0;
  int numberOfRuns = 1000;
  int passed = doPerformanceTest(numberOfRuns, 100);
  double percentagePassed = passed / numberOfRuns;
  assertTrue("percentagePassed = " + percentagePassed, 
percentagePassed >= percentageRequired);
}

This statistical testing technique is quite useful for addressing a number of problems. It is a necessity for testing methods that have results based on dates, times, or random numbers. It's also invaluable when testing for performance and other non-deterministic properties of your code.

Previous Page

Next Page