GrantPalin.com

Introduction

Previously in this series, I’ve shown how to set up and structure a Subversion repository, use NAnt to automate the project build process, and use both FxCop and NDepend to analyze the codebase.

• compile code - NAnt
• run unit tests
• perform code analysis - FxCop, NDepend
• generate documentation
• create distributable packages

Following along with the series, I’ve decided it’s time I added some unit tests to my solution. Following is a kind of step-by-step on how I started the process.

What is Unit Testing?

I will not be going into the basics of unit testing in much depth here, but I think it is important to at least make a mention of it. I will do so in point form here - there are numerous more comprehensive resources, some of which I have linked in the Additional Reading section at the end of this article.

Unit testing:

• tests individual portions (usually classes) of production code, one by one, independently of each other.
• reduces the need for manual, time-consuming, and error-prone testing.
• does not replace manual testing, not by any means. Manual tests will always be needed in some degree.
• is not integration testing, which tests units of code working with one another. Unit testing means testing individual units on their own.
• increases confidence in code under test, and makes changes to such code easier.
• can be run automatically as part of a project build process.
• can be done in some form or another in mostly any programming language.
• has numerous tools available to help the process, again often based on language.

How to do Unit Testing

Unit testing can be done by scripting or by writing code - strange, writing more code to test production code! But there are many unit test tools and frameworks out there that handle a lot of set-up and plumbing work for you. So you can worry about just your tests.

To unit test a piece of code (assuming a class, from now on), you first need to write some tests. Typically, a test class (also called test fixture) is created to match a class in production code. The test class contains individual tests (test cases) that put the matching production class through it’s paces. An individual test case will work on a single method of the class under test. Multiple tests can be written on the same method, if necessary, to account for different outputs based on input, as well as exceptional situations.

Once the tests are written, they need to be run. This is usually where a unit test tool comes in. It essentially just executes the tests you have written and displays the results. There is more to it than that, of course, but there it is in a nutshell.

The above outline applies to unit testing in any language. It is the specific tools, frameworks, and techniques that differ from one language to another. As this series is about .NET, that is the direction in which this post will proceed.

Enough theory, it’s time for some action!

Starting with NUnit

There are multiple unit testing frameworks for .NET, including NUnit, MbUnit, MSTest, and xUnit.Net. As the title of this post indicates, I am using NUnit. Each of the frameworks provides slightly different means to the same end. They tend to differ in syntax and structure, and in the way they are utilized. MSTest is part of Microsoft Visual Studio Team System, and is not available separately. The remaining frameworks are open source, free to use, and available on their respective websites. The xUnit.net website has an interesting feature comparison matrix for these frameworks.

NUnit provides an MSI installer, so it can be easily installed on a computer that will be used for development. However, the ZIP package is more useful as it can be added to a project’s repository, in much the same way as I have done with FxCop and NDepend. This helps to reduce the number of external dependencies to build and run the project, and sidesteps the issue of having the same software installed and configured on each development system.

At the time of this writing, the current stable version is NUnit is 2.4.7. Version 2.5.0 is in alpha, and is not expected to be production-ready for some time yet. The alpha has some interesting new features, but nothing that can’t be lived without 2.4.7 is fairly recent, so it will be fine.

Like with the other tools, NUnit can be extracted to a directory under the tools directory in the project repository. Then add the NUnit directory and files to the repository through the Subversion tool. A later checkout of the repository will bring the NUnit files, and any other dependencies, along with it.

You can now go into your solution in Visual Studio and add a test project. It seems common practice to have a test project to match a class library project, but feel free to set up your test classes/projects how you see fit. The examples here will use the test project per library project layout. For the test project, you will need to add a reference to the NUnit framework - this is the nunit.framework.dll file in the NUnit bin directory. In my case, I am adding tests for the Wadmt.Core project, so the test project is Wadmt.Core.Tests.

Below is a skeleton test fixture class. Notice the use of TestFixture, SetUp, and TearDown attributes. They designate structures with special meaning to NUnit. This is the basic starting point when writing a test class. Add a class to the test project, and add the following starter code (adjust the names as desired):

Imports Wadmt.Core
Imports NUnit.Framework

Namespace Wadmt.Core.Tests

    <TestFixture()> _
    Public Class PrimaryKeyColumnCollectionTest
        Private Fixture As PrimaryKeyColumn

        <SetUp()> _
        Public Sub SetUp()
            Fixture = New PrimaryKeyColumn()
        End Sub 'SetUp

        <TearDown()> _
        Public Sub TearDown()
        End Sub 'TearDown

    End Class

End Namespace

Observe the Imports statement for the Namespace being tested - this provides access to the class beng tested, which is in the indicated namespace. The NUnit framework import is to provide the attributes shown above, as well as various testing functionality - more to come on that. Also notice the use of a member variable, which is named Fixture as it is the test subject.

Note in the code snippets the underscore (_) on some lines - it is the line continuation character in VB.

The SetUp and TearDown methods handle any common initialization and cleanup code for the object under test. These functions are automatically called before and after, respectively, each test case is run. Their names don’t have to be SetUp or TearDown, it is the attributes that matter. Also note the Imports statement for the namespace where the subject class is located. This is so the test fixture can create an instance of the subject class.

Following is a block of code outlining a class that needs some testing. It has some fields, a constructor, and some properties. Not very complicated, but it serves as a starting point for unit testing this project. Note that the constructor and properties have been placed, but there is just dummy code within the properties. There is a reason for this, so read on.

Imports System.Collections

Namespace Wadmt.Core

    Public Class PrimaryKeyColumn
        Inherits DisplayColumn

        Private _Values As ArrayList
        Private _DisplayElement As String
        Private _DisplayElementConstraint As String

        Public Sub New()
            MyBase.New()
            _Values = New ArrayList()
        End Sub

        Public Sub New(ByVal Name As String, ByVal table As String, ByVal label As String)
            MyBase.New(Name, table, label)
            _Values = New ArrayList()
        End Sub

        Public Property DisplayElement() As String
            Get
                Return ""
            End Get
            Set(ByVal Value As String)
                _DisplayElement = ""
            End Set
        End Property

        Public Property DisplayElementConstraint() As String
            Get
                Return ""
            End Get
            Set(ByVal Value As String)
                _DisplayElementConstraint = ""
            End Set
        End Property

        Public Property Values() As ArrayList
            Get
                Return Nothing
            End Get
            Set(ByVal Value As ArrayList)
                _Values = Nothing
            End Set
        End Property

    End Class

End Namespace

I won’t go into the purpose of this class, as that is not as important as testing it. By looking at the above code, you get an idea of what the test fixture needs to do. It’s time to add some test cases to the test fixture. Note the use of the Test attribute - it tells NUnit that the marked method is a test case.

Imports Wadmt.Core
Imports NUnit.Framework

Namespace Wadmt.Core.Tests

    <TestFixture()> _
    Public Class PrimaryKeyColumnTest
        Private Fixture As PrimaryKeyColumn

        <SetUp()> _
        Public Sub SetUp()
            Fixture = New PrimaryKeyColumn("column", "table", "label")
        End Sub

        <TearDown()> _
        Public Sub TearDown()
        End Sub

        <Test()> _
        Public Sub NewColumnHasNoValues()
            Assert.AreEqual(0, Fixture.Values.Count)
        End Sub

        <Test()> _
        Public Sub AddingValueIncrementsValueCount()
            Fixture.Values.Add("new value")
            Assert.AreEqual(1, Fixture.Values.Count)
        End Sub

        <Test()> _
        Public Sub NewColumnFormElementIsNull()
            Assert.IsNull(Fixture.DisplayElement)
        End Sub

        <Test()> _
        Public Sub NewColumnConstraintIsNull()
            Assert.IsNull(Fixture.DisplayElementConstraint)
        End Sub

    End Class

End Namespace

Now it’s time to get a look at the NUnit GUI. Browse to the NUnit bin folder and double-click nunit.exe to run the app.

At this point, it is just an empty window. You need to create a new project, and add the test project here - the .dll containing your test fixtures (if more than one, add them all). In this case, I added the Wadmt.Core.Tests assembly. Save the NUnit project in an easy to find location, typically close to your actual project. I saved the wadmt.nunit file within my project’s root directory, and added it to the repository.

With the target added (Wadmt.Core here), just click the arrow and see the result. In this case, it’s all red, as the tests are running, but they are not passing - remember the dummy class implementation!

Now that it is known which tests are failing, just go into your class and write code that will cause the tests to pass. In the case of the example, it’s just a matter of revising the dummy properties so the tests will get the expected results. The completed class implementation is shown below.

Imports System.Collections

Namespace Wadmt.Core

    Public Class PrimaryKeyColumn
        Inherits DisplayColumn

        Private _Values As ArrayList
        Private _DisplayElement As String
        Private _DisplayElementConstraint As String

        Public Sub New()
            MyBase.New()
            _Values = New ArrayList()
        End Sub

        Public Sub New(ByVal Name As String, ByVal table As String, ByVal label As String)
            MyBase.New(Name, table, label)
            _Values = New ArrayList()
        End Sub

        Public Property DisplayElement() As String
            Get
                Return _DisplayElement
            End Get
            Set(ByVal Value As String)
                _DisplayElement = New String(Value)
            End Set
        End Property

        Public Property DisplayElementConstraint() As String
            Get
                Return _DisplayElementConstraint
            End Get
            Set(ByVal Value As String)
                _DisplayElementConstraint = New String(Value)
            End Set
        End Property

        Public Property Values() As ArrayList
            Get
                Return _Values
            End Get
            Set(ByVal Value As ArrayList)
                _Values = Value
            End Set
        End Property

    End Class

End Namespace

When that is done, the project can be recompiled, and you can return to the NUnit window. Rerun the tests. If they all pass - as shown below - then great. If not, review your code for any possible failure points, and correct them.

Repeat the process as necessary to have the class and all methods and properties under test.

Integrating NUnit into NAnt Builds

At this point, some tests have been written, and run with the NUnit GUI application. The GUI is a nice visual way to check the status of tests, but it is an extra step in what should be an automated build. Fortunately, like the other tools used so far (FxCop, NDepend), NUnit provides a command-line utility to run tests and display the results.

This command-line tool can be used by adding the following code block to the project’s build file.

<target name="test" depends="build.app, build.tests, run.tests" description="builds and runs unit tests" />

<target name="build.tests">
    <!-- compile the Core.Tests assembly -->
    <vbc target="library" output="${dir.build.bin}\Wadmt.Core.Tests.dll" debug="${debug}">
        <sources>
            <include name="${dir.src}\Wadmt.Core.Tests\***.vb" />
        </sources>
        <references>
            <include name="System.dll" />
            <include name="${dir.build.bin}\Wadmt.Core.dll" />
            <include name="${dir.tools}\nunit\bin\nunit.framework.dll" />
        </references>
    </vbc>
</target>

<target name="run.tests">
    <property name="dir.reports.nunit" value="${dir.reports}\NUnit\" />

    <mkdir dir="${dir.reports.nunit}" unless="${directory::exists(dir.reports.nunit)}" />

    <property name="nunit.output" value="${dir.reports.nunit}\nunit-results.xml" />
    <property name="nunit.input" value="wadmt.nunit" />

    <exec program="${dir.tools}\nunit\bin\nunit-console.exe" failonerror="true">
        <arg value="${nunit.input}" />
        <arg value="/xml:${nunit.output}" />
    </exec>
</target>

So above are three tasks that can be added to a build file. The first is just an empty task that requires the other two be completed, in order, before this task is complete. The second task builds the unit tests, and the third handles the running of the tests. I think it makes sense to have them separated this way, so that each task only has one objective.

Nothing special is done in the compilation step, just compiling code with references, same as with compiling any other code. The output assembly needs to have the same name as what is recorded in the NUnit project file, so there will be no confusion. The run step just executes the NUnit command-line program, passing as parameters the NUnit project file, and the location of an output report. The report will always be in XML format, hence the filename. The NUnit documentation provides many more command-line options, but so far this setup suits me fine.

With the tests written, and the NAnt tasks in place, one just needs to enter the following line into a command window at the project root:

build test

And below is a copy of the output of the command.

run.tests:

[exec] NUnit version 2.4.7
[exec] Copyright (C) 2002-2007 Charlie Poole.
[exec] Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov.
[exec] Copyright (C) 2000-2002 Philip Craig.
[exec] All Rights Reserved.
[exec] Runtime Environment -
[exec]    OS Version: Microsoft Windows NT 5.1.2600.0
[exec]   CLR Version: 1.1.4322.2407 ( Net 1.1.4322.2407 )
[exec] ....
[exec] Tests run: 4, Failures: 0, Not run: 0, Time: 0.188 seconds

Moving Forward

Some points have been made, techniques demonstrated. But there is so much more to unit testing than has been shown in this article.

For one, there is a higher level to unit testing: integration testing. The difference is that unit tests exercise individual components of your solution; integration testing, on the other hand, tests the interaction between the different units. So unit tests ensure that the individual parts work alone as expected, and integration tests ensure that the parts work together.

Next, for the technique to work, it is important to ensure unit tests get written and run. It is best that the tests cases are written before the code, since the tests become a type of design specification. And as time goes on, the tests also serve as documentation. This is a nice side effect.

Writing the tests cases after the code has been written is a different mindset. One has to be careful to ensure that the tests are not being written so that the current code will pass them. Plus, it’s a bit of a pain. I’ve been feeling that with my WADMT project, which is of some size, and has quite a few parts that need tests.

And that brings up another point regarding adding tests to existing projects. When adding tests, there is the option of doing it wholesale - that is, doing a big whack of tests all at once. But that is not very effective, and would get tedious very quickly. I think the better way to do it is while doing work on production code.

Suppose you are making an addition to your code, or refactoring, or some other change. Write the relevant test cases first, then do the edits. Ensure the tests pass. Repeat as necessary. The theory is that given enough time, and enough changes, the project will eventually have good test coverage.

Speaking of which, there are some more advanced topics related to unit testing, including mocking and test coverage. Those are worthy of their own articles, and are beyond the scope of this intro. They will be covered at a later time.

Conclusion

So, here we are. I briefly outlined the practice of unit testing in general, and the benefits the process provides. I then demonstrated how to utilize NUnit to build and run unit tests for an existing .NET project. Finally, I then finished with a brief look at the different ways in which unit testing can be utilized beyond basic tests.

Even having just a fraction of my project’s codebase under test, the automated tests reinforce my confidence in the code. For my WADMT solution, as long as I write some tests for any code I will be changing, BEFORE making the changes, the project will eventually have a good portion under test. And for new projects, I will strive to write the tests first, then write code that pass those tests.

The test-first mindset is rather different from test-after, and is something I will need some getting used to. But I can already see the benefits to be had in doing so!

Bottom line: unit testing is a practice which requires an investment of time and effort up front, but which will pay for itself as a project progresses. This practice is not specific to any one programming language, although numerous tools and frameworks are available to assist the process, regardless of language.

Additional Reading

• Test Driven Development Using NUnit in C# - good overview of NUnit by 4 Guys from Rolla
• Avoid retrofitting unit tests - discussion on adding unit tests to existing solutions
• We don’t write tests. There just isn’t time for luxuries. - A good look into the investing of time and effort into unit testing, and the resulting benefits
• Unit Test Project Structure Poll - poll and discussion on how to organize unit tests
• Unit Test Projects or Not? - follow-up discussion regarding the project structure poll