Shortcomings of Unit Testing
No product or process is a silver bullet, and unit tests are no different. Using the unit testing process and frameworks described in this article, you should also consider several shortcomings when deciding how to test your code.
Attributes of Effective Unit Tests
- Can you effectively test code that uses expensive external resources (such as a database, a mainframe, or a fax server)?
- Can you test units that depend on other units?
- How will you depth-test objects?
- Do you know how to verify the internal state of an object without violating encapsulation?
- How can you test against code that has not been written yet?
- What if the tests being written are not enough to completely test the unit? Or what if the tests themselves are not correct?
Writing effective unit tests requires practice. The tests should exercise enough of the possible inputs of a function without testing everything. Here are several attributes of effective unit testing that you should keep in mind when you're writing unit tests. These attributes are similar to the ACID attributes of transactions.
Atomic. All assertions within each unit test should pass, or the entire test should fail. Within one logical test you may have several different assertions. For example, you might be trying to test an entire process, where changes to several different components occur. You want to verify all changes before declaring the test successful.
Consistent. Results of a test run should remain the same as long as the inputs remain the same. Sometimes objects can keep track of state internally, when this happens, you should make sure the state remains the same between calls. If any assertions fail when running the same method call over and over again, then that means there are internal properties that are not being cleared.
Isolated. Tests should be isolated from each other and be able to be run in any order. That means you should be able to test any part of the system without having to set up a separate part of the system. As a concrete example, you should need to run test method testA() in order for a separate test method, testB(), to pass.
Durable. Tests should be resilient enough so that all other tests run, regardless of previous tests failures. If you have a suite of 10,000 tests and one fails, then the other tests should continue to run, without knowledge of any previous failures. The result of one test should not affect the outcome of any other test. For example, all tests should cleanup after themselves, including removing any files created or any database records added. This can be tested by immediately running your tests twice, just to make sure that there are no residual effects of the tests being left behind.
Automated unit testing is an essential tool for any programmer. Your code will become simpler and you will find yourself spending less time debugging and more time playing foosball! Additionally, the ability to change and refactor your code while knowing that you have a built-in safety net in the form of automated unit tests provides a huge advantage.