Improve Code Quality with Unit Testing in Visual Studio Team Edition

any Agile development methodologies require testing source code early and often. Because inefficient software testing methods result in code that is not tested often, it is critical to adopt a testing strategy that is both as automated as possible and also lets developers author tests easily and quickly. Visual Studio Team Edition provides software developers with an integrated tool that works well not only with Visual Studio but also with the supporting Team Foundation Server system of tools.

Unit tests provide the most basic way to automatically test software. Development teams using Test Driven Development (TDD) write unit tests before writing the code, which helps to define when the software is complete?when the software passes all the tests, it’s complete. Taking this paradigm a step further, code that is not exercised by the unit tests is superfluous, because it’s not necessary to satisfy the software requirements. Other development teams write unit tests after completing development, using the tests as a way to verify that the code still operates properly after future changes to the software.

Other than just testing the software, another key feature of a good unit testing framework is code coverage analysis. After running a battery of unit tests against a piece of software, you should be able to view the original source and see which lines of code were exercised during the test. If you find code that was not tested, you should consider either adding unit tests to exercise the untested code?or if you’re using TDD, removing it.

There are many unit testing frameworks available for Visual Studio. Most notably, the widely used NUnit works with all .NET languages. You write NUnit tests in much the same way as Visual Studio unit tests. Other commercially available unit testing solutions available include tools that supplement the features of Visual Studio’s unit testing tools. While not best of breed, the unit testing tools provided by Visual Studio are a great way to get started with unit testing your .NET code.

?
Figure 1. The DevXLibrary Project in Solution Explorer: The figure shows the DevXLibrary project containing the ProjectLibrary class, used in this article as an example for the unit tests.

The Sample Project
As an example, I have provided a simple project named DevXLibrarycontaining a single class named ProjectLibrary (see Figure 1). The ProjectLibrary class contains three methods, each of which is intended to show how different areas of the unit testing features work. Each method could be used to validate a person’s social security number using rules and information published by the Social Security Administration.

Author’s Note: If you are interested, the information used to code these examples was found on two Web pages: the Wikipedia entry for Social Security Numbers and the Social Security Administration’s Web page that maps social security numbers to their likely state of origin.

Build Your First Unit Test
The ProjectLibrary class in the sample project defines three simple methods.

  • SSN_IsValidLength simply checks the length of an SSN parameter to ensure that the string contains nine characters.
  • SSN_IsValid is a more complex verification of the SSN against a set of rules for SSNs.
  • SSN_StateOfIssue uses the SSN to find the state where it was likely issued, using the first 3 digits of the SSN to find a matching state. This method could have been coded against a database, but coding it as a large select case made it a great example for this exercise. Further, rather than using Shared methods to make interaction with the class simpler, I chose to use instance methods to demonstrate another feature of the unit testing framework.

Here’s the code for SSN_IsValidLength.

?
Figure 2. Creating Unit Test Boilerplate Code: In Visual Studio, right click the name of the method you wish to test and then select “Create Unit Tests.”
   ' Verifies that an SSN is the valid length   Public Function SSN_IsValidLength(ByVal SSN As String) _      As Boolean         Dim blnReturn As Boolean      If String.IsNullOrEmpty(SSN) Then         ' an SSN must be provided         blnReturn = False      ElseIf SSN.Length <> 9 Then         ' an SSN must be 9 numbers long         blnReturn = False      Else         ' the length must be correct         blnReturn = True      End If      Return blnReturn   End Function

To create a unit test for this method, right click the method name and select “Create Unit Tests?,” as shown in Figure 2.

?
Figure 3. The Create Unit Tests Dialog: Using this dialog, you can select the methods for which you want to create unit tests automatically.
?
Figure 4. The Test Generation Settings Dialog: This dialog provides control over test project creation and boilerplate test code generation.
?
Figure 5. Specify Output Language: The Create Unit Tests dialog allows you to specify whether the unit tests should be generated using C# or Visual Basic.
?
Figure 6. Generated Unit Test Project: The system generates the project and adds it to the Solution Explorer.

Figure 3 shows the Create Unit Tests dialog and all the methods available in the ProjectLibrary class. Select the checkbox for each method for which you wish to create unit tests (all three methods in this example).

Click the “Settings?” button to open the “Test Generation Settings” dialog as shown in Figure 4.

This dialog provides several options that determine how you want the system to write the test class and boilerplate unit test code. I have modified the default settings only to add an underscore character before the word “Test” in the file name, class name, and method names of the resulting code. As shown in Figure 5, the Create Unit Tests dialog also allows you to specify whether to generate unit tests in C# or Visual Basic.

After clicking “OK,” you will be prompted for the name of the new unit testing project. When you click “Create” in this dialog, Visual Studio creates the unit test project, and adds the resulting project to the Solution Explorer as shown in Figure 6.

Working with the Generated Code
In addition to the ProjectLibrary_Test.vb file, which contains the unit test boilerplate code, you’ll find several other files.

  • AuthoringTests.txt contains instructions on how to use the unit testing features of Visual Studio.
  • localtestrun.testrunconfig contains configuration information used by the testing framework. Double-clicking this file opens a window where you can modify the configuration settings for the test.
  • DevXLibrary.vsmdi contains details about the tests you can run. Double-clicking this file opens the Test View pane where you can launch tests.

Visual Studio creates one stub test method for each method selected in the Create Unit Tests dialog. Each stub code test method includes class creation, variables for input parameters and return values, and code to compare the return value to what was expected. If the methods being tested were declared as Shared methods, the code would not include class initialization and would call the method directly. An Assert verifies that the method’s return value matches the expected value. If the values do not match, the Assert raises an error that causes the test to fail. Also notice that Visual Studio adds a call to Assert.Inconclusive (with the message “Verify the correctness of this test method.”) to the end of all unit test methods it creates. This call alerts the tester that the unit test method has not been implemented. After you have properly implemented the unit test method you should remove this line of code.

Here’s the code generated for the SSN_IsValidLength method.

   '''   '''A test for SSN_IsValidLength(ByVal String)   '''    _   Public Sub SSN_IsValidLength_Test()       Dim target As ProjectLibrary = New ProjectLibrary          Dim SSN As String = Nothing 'TODO: Initialize to an appropriate value          Dim expected As Boolean       Dim actual As Boolean          actual = target.SSN_IsValidLength(SSN)          Assert.AreEqual(expected, actual,           "DevXLibrary.ProjectLibrary.SSN_IsValidLength did not return " & _          "the expected value.")       Assert.Inconclusive("Verify the correctness of this test method.")   End Sub

To make this unit test method work correctly, you need to modify the unit test method in several ways. First, set the SSN variable to the value you want this unit test to use for the SSN parameter. Next, set the expected variable to the expected return value of the SSN_IsValidLength when given the SSN parameter. Finally, remove the call to Assert.Inconclusive. The resulting SSN_IsValidLength unit test method should look like this:

    _    _   Public Sub SSN_IsValidLength_Short_Test()         Dim target As ProjectLibrary = New ProjectLibrary      Dim SSN As String = "12345678"      Dim expected As Boolean = False      Dim actual As Boolean         actual = target.SSN_IsValidLength(SSN)   
?
Figure 7. The Test Results Pane: This pane displays the results of the unit tests. To get the description column to display, right click the column headings row, select "Add/Remove Columns," and add the Description field.
Assert.AreEqual(expected, actual, "DevXLibrary.ProjectLibrary.SSN_IsValidLength did " & _ "not return the expected value.") End Sub

If you look closely at the preceding code, you will notice that other changes to the boilerplate code have been made as well. I chose to test the SSN_IsValidLength method by providing a value for SSN with fewer than the required nine characters. To distinguish this test from other tests that might exercise the same method, I gave the test method a distinguishing name: SSN_IsValidLength_Short_Test. I also set an additional attribute on the method named Description. This attribute provides more information about what the method actually tests.

In the downloadable source for this project, I also added two additional tests for the ProjectLibrary’s SSN_IsValidLength method named using this naming convention. One tests for SSNs longer than nine characters (SSN_IsValidLength_Long_Test), and the other verifies that the method properly recognizes a valid SSN with nine characters (SSN_IsValidLength_Correct_Test).

To run the tests, click the test project in Solution Explorer and then select “Start Selected Test Project with Debugger” from Visual Studio’s Test menu. The Test Results pane opens, displaying the results of the unit tests. Figure 7 shows that all three tests passed, along with the names and descriptions of the test methods. If a unit test fails, the result column displays a red failed message, along with a basic error message describing the problem.

?
Figure 8. A Failed Test: Here’s the Test Results Pane showing a failed test.
Author’s Note: The Description column does not appear by default in the Test Results and Test View panes. To display the test descriptions, right click the column heading row, and select “Add/Remove Columns?.” Select the Description field and click OK. You will need to perform this same procedure for both the Test View and Test Results panes. Adding the Description column aids greatly in identifying tests and results quickly.

Figure 8 shows the Test Results pane following a failed test.

When you start unit tests as described above, the system runs all the tests in the test project. When you don’t want to run all the tests, open the Test View pane by using the Test/Windows/Test View menu item. The Test View pane shows each of the tests contained in the test project, along with each tests’ description. To execute specific tests, select the tests you want to run, and click the Run button in the upper left corner.

Database-driven Unit Tests
So far, you’ve seen simple tests implemented by providing all the input and expected output values in code; however, in many the number of test cases is too numerous to implement by writing a separate test method for each test case. One of the methods included in the ProjectLibrary class is named SSN_StateOfIssue. This method returns the most likely name of a state or US Territory where a SSN was issued, based on information from the Social Security Administration. The method looks at the first three digits of an SSN, matches it to a long list of assignments, and finds the state associated with the SSN. Methods like this one are great candidates for database-driven unit tests.

The test works similarly to the non-database driven tests, but there are a few differences that make these tests unique. The first major difference is the addition of a Datasource attribute on the test method, which tells the unit testing framework where to find the data for the test and which table to open. The second major difference is that the system calls the data-driven test method once for every row in the table specified by the Datasource attribute. Lastly, the unit testing framework provides a standard way of retrieving the values from the current data row in the test table. In database driven unit tests, you access the current database row using the TestContext.DataRow function. The fields in the data row are used by your test method not only for values for the input parameters to your method but also for storing the expected return value of the method. The call to Assert.AreEqual works the same as the non-data driven unit tests. Here’s the implementation of the SSN_StateOfIssue_Test method.

    _    _   Public Sub SSN_StateOfIssue_Test()   
?
Figure 9. Test Table: You can format the table used for building a data-driven unit in whatever manner is required; Visual Studio places no constraints on the data format.
Dim target As ProjectLibrary = New ProjectLibrary Dim SSN As String Dim expected As String Dim actual As String SSN = TestContext.DataRow("SSN").ToString expected = TestContext.DataRow("ExpectedValue").ToString actual = target.SSN_StateOfIssue(SSN) Assert.AreEqual(expected, actual, "DevXLibrary.ProjectLibrary.SSN_StateOfIssue did " & _ "not return the expected value.") End Sub

In the data driven unit test example above, I created a Microsoft Access database with a single table cleverly named SSN_StateOfIssue_Test. The table contains an auto-incrementing primary key value, an SSN field used for input to the method, and an expected return value. You can see the Access database in the downloadable code for this article. Figure 9 shows the organization of the data used by the SSN_StateOfIssue_Test unit test.

?
Figure 10. Data-driven Test Results The test results show that the test failed on the sixth row of test data. For the purposes of this test, Puerto Rico was intentionally misspelled in the test database.

Open the Test View pane, and run the SSN_StateOfIssue_Test test. You will find that the test fails. To demonstrate the effect of a failed test, I have purposefully entered an incorrect state name into the testing data. Look again at Figure 9 and notice that in row id 6, Puerto Rico is misspelled. Even thought the Test Results pane shows that the test failed, it does not show an error message, because in a data driven test, multiple rows of test data might have caused the test to fail. To find out which rows failed the test, right-click the failed row in the Test Results pane and select “View Test Results Details.” The window that opens shows the details of the test, including the results of the unit test for each data row in the test data table. Figure 10 shows that the test failed with the data from row 5. Notice also that the cause of the test failure is shown in the Error Message column.

Code Coverage Analysis
So far, you’ve seen how to build a series of unit tests that completely exercises all the code in the class. You’ve built multiple tests for methods that required more testing, and even built a database table full of test data to exercise the SSN_StateOfIssue method. But how can you be certain that you’re testing all the code? Must

?
Figure 11. Code Coverage Instrumentation: Select the target class (the class being tested) for code coverage instrumentation. You will not need to select the test class.

you manually verify that the tests provide sufficient usage scenarios to ensure that the code works? Fortunately, you don’t have to do that. Visual Studio provides a code coverage analysis tool. That is, when you start your test, the unit testing framework keeps track of which lines of code were run during the test. After the test, you can inspect the class for lines of code that were not executed during the test. Then, you can choose to either remove the unused lines of code or add tests to ensure that the untested code gets tested.

You must first configure the class for code coverage analysis. In the Solution Explorer, double-click the localtestrun.testrunconfig file to open the test configuration window. Select “Code Coverage” in the list on the left. As shown in Figure 11, select the DevXLibrary.dll to instrument for code coverage. Click Close and return to the Test View pane.

Select the three test methods for SSN_IsValidLength and select “Run Selection” from the Test View pane’s Run button (see Figure 12).

It is important that you do not choose “Debug Selection” because code coverage analysis does not work when the class is run in debug mode. When the test is complete, right-click any of the Passed test rows in the Test Results pane, and select “Code Coverage Results.” The Code Coverage Results pane shown in Figure 13 will open and show the code coverage statistics resulting from the selected test.

?
Figure 12: The Run Dropdown Button: Dropping the Run button’s dropdown pane lets you select between running and debugging the selected methods.
?
Figure 13. Code Coverage Statistics: The Code Coverage Results pane displays the code coverage statistics.
?
Figure 14. Source Code Coverage: In the source code view of code coverage, the blue lines indicate lines that ran during the tests, while the red lines indicate lines that were not tested.

If you inspect the results of the code coverage analysis, you will see that the tests failed to exercise the code in the SSN_IsValid and SSN_StateOfIssue tests; but tested nearly all of the code in the SSN_IsValidLength test?because I told you to select only the SSN_IsValidLength tests to run above. To see a more accurate view of the code coverage for the other two methods, you must select them when running the test.

To find out which lines of code did not get tested, right click the SSN_IsValidLength method in the Code Coverage Results pane and select “Go to source code.” Figure 14 shows the resulting source code view. In this view, the blue lines of code were executed during the test while the red lines were not executed. Using this information, you will be better able to modify your tests to exercise your code more completely.

Test All the Code
In most cases, even simple methods will require several test methods to completely exercise all the functionality of the method. It is helpful to give each of the test methods names that help identify what area of functionality is being tested, as well as to provide a description attribute containing a concise explanation of the test method. The unit tests that were created to test the SSN_IsValidLength method are good candidates for this type of additional ‘documentation’.

Rather than creating multiple test methods, another approach to testing a method is to put all the tests into a single test method. Depending on what is being tested and your own coding preferences, this may be a more convenient way of organizing your tests. To demonstrate this method of organizing your tests, I implemented the tests for SSN_IsValid by including all the tests in a single test method named SSN_IsValid_Test (see Listing 1), which contains a series of calls to test methods and a call to Assert.AreEqual after each to verify the results of the method call.

There are disadvantages to putting all your tests in a single test method. One problem is that it is more difficult to manage running of individual tests?they are executed as a single unit. In addition, if one of the tests fails, subsequent tests are not run, which may force you to cycle through running the test and repairing bugs several times before the test passes. In contrast, when you implement each test as a separate method, the system can run all the tests independently, showing you all the problems at the same time.

Subs (void functions) vs. Methods with Return Values
In the event that the method being tested is a Sub (a void function in C#), which has no return value, the boilerplate code includes a call to Assert.Inconclusive with the following message: “A method that does not return a value cannot be verified.” To test methods that do not have a return value, you must provide code to verify that the method performed as expected. For example, if the method updates a row in a database, you would need to provide code to verify that the expected changes were made successfully.

Initialization and Cleanup
Testing methods that modify databases can become cumbersome since each run of the test may prevent the test from running successfully on later runs. To help with this problem, the unit test framework provides two ‘methods,’ ClassInitialize and ClassCleanup, which are called before any of the tests are run and after all the tests have completed, respectively. ClassInitialize and ClassCleanup are not really methods?they are attributes you add to a method in your test class to signal to the framework which methods to call to perform the ClassInitialize and ClassCleanup functions.

To help with preparing a database for unit testing, place code in the ClassInitialize method that ensures the appropriate records are present, and that they have the beginning values the unit tests expects. Likewise, to enable future tests to be run, use the ClassCleanup method to reset any rows that were modified during the unit test.

Similarly, you can also use the ClassInitialize and ClassCleanup methods to construct complex objects that are required by the tests. The test methods created by Visual Studio provide code to simply create the class being tested, but in many cases, the class being tested will require properties to be set, initialization methods to be called, etc. In these cases, either provide a separate method to prepare the target class that can be called by each test method, or put such code in the ClassInitialize method.

The unit tests presented in this article should help you get comfortable with experimenting with unit testing in Visual Studio. The unit testing framework supported by Visual Studio is much more comprehensive than could be covered appropriately in this article. I encourage you to experiment with unit testing in your own projects, and try the integrated tools Visual Studio provides.

Share the Post:
Share on facebook
Share on twitter
Share on linkedin

Overview

The Latest

microsoft careers

Top Careers at Microsoft

Microsoft has gained its position as one of the top companies in the world, and Microsoft careers are flourishing. This multinational company is efficiently developing popular software and computers with other consumer electronics. It is a dream come true for so many people to acquire a high paid, high-prestige job

your company's audio

4 Areas of Your Company Where Your Audio Really Matters

Your company probably relies on audio more than you realize. Whether you’re creating a spoken text message to a colleague or giving a speech, you want your audio to shine. Otherwise, you could cause avoidable friction points and potentially hurt your brand reputation. For example, let’s say you create a

chrome os developer mode

How to Turn on Chrome OS Developer Mode

Google’s Chrome OS is a popular operating system that is widely used on Chromebooks and other devices. While it is designed to be simple and user-friendly, there are times when users may want to access additional features and functionality. One way to do this is by turning on Chrome OS