Database-driven Unit Tests
So far, you've seen simple tests implemented by providing all the input and expected output values in code; however, in many the number of test cases is too numerous to implement by writing a separate test method for each test case. One of the methods included in the ProjectLibrary class is named SSN_StateOfIssue
. This method returns the most likely name of a state or US Territory where a SSN was issued, based on information from the Social Security Administration. The method looks at the first three digits of an SSN, matches it to a long list of assignments, and finds the state associated with the SSN. Methods like this one are great candidates for database-driven unit tests.
The test works similarly to the non-database driven tests, but there are a few differences that make these tests unique. The first major difference is the addition of a Datasource
attribute on the test method, which tells the unit testing framework where to find the data for the test and which table to open. The second major difference is that the system calls the data-driven test method once for every row in the table specified by the Datasource
attribute. Lastly, the unit testing framework provides a standard way of retrieving the values from the current data row in the test table. In database driven unit tests, you access the current database row using the TestContext.DataRow
function. The fields in the data row are used by your test method not only for values for the input parameters to your method but also for storing the expected return value of the method. The call to Assert.AreEqual
works the same as the non-data driven unit tests. Here's the implementation of the SSN_StateOfIssue_Test
"Provider=Microsoft.Jet.OleDb.4.0; Data Source=""c:\Test.mdb""", _
"SSN_StateOfIssue_Test", DataAccessMethod.Sequential)> _
Public Sub SSN_StateOfIssue_Test()
Dim target As ProjectLibrary = New ProjectLibrary
Dim SSN As String
Dim expected As String
Dim actual As String
SSN = TestContext.DataRow("SSN").ToString
expected = TestContext.DataRow("ExpectedValue").ToString
actual = target.SSN_StateOfIssue(SSN)
"DevXLibrary.ProjectLibrary.SSN_StateOfIssue did " & _
"not return the expected value.")
|Figure 9. Test Table: You can format the table used for building a data-driven unit in whatever manner is required; Visual Studio places no constraints on the data format.|
In the data driven unit test example above, I created a Microsoft Access database with a single table cleverly named SSN_StateOfIssue_Test
. The table contains an auto-incrementing primary key value, an SSN field used for input to the method, and an expected return value. You can see the Access database in the downloadable code for this article. Figure 9
shows the organization of the data used by the SSN_StateOfIssue_Test
|Figure 10. Data-driven Test Results The test results show that the test failed on the sixth row of test data. For the purposes of this test, Puerto Rico was intentionally misspelled in the test database.|
Open the Test View pane, and run the SSN_StateOfIssue_Test
test. You will find that the test fails. To demonstrate the effect of a failed test, I have purposefully entered an incorrect state name into the testing data. Look again at Figure 9
and notice that in row id 6, Puerto Rico is misspelled. Even thought the Test Results pane shows that the test failed, it does not show an error message, because in a data driven test, multiple rows of test data might have caused the test to fail. To find out which rows failed the test, right-click the failed row in the Test Results pane and select "View Test Results Details." The window that opens shows the details of the test, including the results of the unit test for each data row in the test data table. Figure 10
shows that the test failed with the data from row 5. Notice also that the cause of the test failure is shown in the Error Message column.
Code Coverage Analysis
So far, you've seen how to build a series of unit tests that completely exercises all the code in the class. You've built multiple tests for methods that required more testing, and even built a database table full of test data to exercise the SSN_StateOfIssue
method. But how can you be certain that you're testing all
the code? Must
|Figure 11. Code Coverage Instrumentation: Select the target class (the class being tested) for code coverage instrumentation. You will not need to select the test class.|
you manually verify that the tests provide sufficient usage scenarios to ensure that the code works? Fortunately, you don't have to do that. Visual Studio provides a code coverage analysis tool. That is, when you start your test, the unit testing framework keeps track of which lines of code were run during the test. After the test, you can inspect the class for lines of code that were not executed during the test. Then, you can choose to either remove the unused lines of code or add tests to ensure that the untested code gets tested.
You must first configure the class for code coverage analysis. In the Solution Explorer, double-click the localtestrun.testrunconfig
file to open the test configuration window. Select "Code Coverage" in the list on the left. As shown in Figure 11
, select the DevXLibrary.dll to instrument for code coverage. Click Close and return to the Test View pane.
Select the three test methods for SSN_IsValidLength
and select "Run Selection" from the Test View pane's Run button (see Figure 12
It is important that you do not choose "Debug Selection" because code coverage analysis does not work when the class is run in debug mode. When the test is complete, right-click any of the Passed test rows in the Test Results pane, and select "Code Coverage Results." The Code Coverage Results pane shown in Figure 13
will open and show the code coverage statistics resulting from the selected test.
|Figure 12: The Run Dropdown Button: Dropping the Run button's dropdown pane lets you select between running and debugging the selected methods.||
|Figure 13. Code Coverage Statistics: The Code Coverage Results pane displays the code coverage statistics.||
|Figure 14. Source Code Coverage: In the source code view of code coverage, the blue lines indicate lines that ran during the tests, while the red lines indicate lines that were not tested.||
If you inspect the results of the code coverage analysis, you will see that the tests failed to exercise the code in the SSN_IsValid
tests; but tested nearly all of the code in the SSN_IsValidLength
testbecause I told you to select only the SSN_IsValidLength
tests to run above. To see a more accurate view of the code coverage for the other two methods, you must select them when running the test.
To find out which lines of code did not get tested, right click the SSN_IsValidLength
method in the Code Coverage Results pane and select "Go to source code." Figure 14
shows the resulting source code view. In this view, the blue lines of code were executed during the test while the red lines were not executed. Using this information, you will be better able to modify your tests to exercise your code more completely.
Test All the Code
In most cases, even simple methods will require several test methods to completely exercise all the functionality of the method. It is helpful to give each of the test methods names that help identify what area of functionality is being tested, as well as to provide a description attribute containing a concise explanation of the test method. The unit tests that were created to test the SSN_IsValidLength
method are good candidates for this type of additional 'documentation'.
Rather than creating multiple test methods, another approach to testing a method is to put all the tests into a single test method. Depending on what is being tested and your own coding preferences, this may be a more convenient way of organizing your tests. To demonstrate this method of organizing your tests, I implemented the tests for SSN_IsValid
by including all the tests in a single test method named SSN_IsValid_Test
(see Listing 1
), which contains a series of calls to test methods and a call to Assert.AreEqual
after each to verify the results of the method call.
There are disadvantages to putting all your tests in a single test method. One problem is that it is more difficult to manage running of individual teststhey are executed as a single unit. In addition, if one of the tests fails, subsequent tests are not run, which may force you to cycle through running the test and repairing bugs several times before the test passes. In contrast, when you implement each test as a separate method, the system can run all the tests independently, showing you all the problems at the same time.
Subs (void functions) vs. Methods with Return Values
In the event that the method being tested is a Sub (a void function in C#), which has no return value, the boilerplate code includes a call to Assert.Inconclusive
with the following message: "A method that does not return a value cannot be verified." To test methods that do not have a return value, you must provide code to verify that the method performed as expected. For example, if the method updates a row in a database, you would need to provide code to verify that the expected changes were made successfully.
Initialization and Cleanup
Testing methods that modify databases can become cumbersome since each run of the test may prevent the test from running successfully on later runs. To help with this problem, the unit test framework provides two 'methods,' ClassInitialize
, which are called before any of the tests are run and after all the tests have completed, respectively. ClassInitialize
are not really methodsthey are attributes you add to a method in your test class to signal to the framework which methods to call to perform the ClassInitialize
To help with preparing a database for unit testing, place code in the ClassInitialize
method that ensures the appropriate records are present, and that they have the beginning values the unit tests expects. Likewise, to enable future tests to be run, use the ClassCleanup
method to reset any rows that were modified during the unit test.
Similarly, you can also use the ClassInitialize
methods to construct complex objects that are required by the tests. The test methods created by Visual Studio provide code to simply create the class being tested, but in many cases, the class being tested will require properties to be set, initialization methods to be called, etc. In these cases, either provide a separate method to prepare the target class that can be called by each test method, or put such code in the ClassInitialize
The unit tests presented in this article should help you get comfortable with experimenting with unit testing in Visual Studio. The unit testing framework supported by Visual Studio is much more comprehensive than could be covered appropriately in this article. I encourage you to experiment with unit testing in your own projects, and try the integrated tools Visual Studio provides.