Mono NUnit Test Guidelines and Best Practices

Authors: Nick Drochak  <ndrochak@@gol.com>
	 Martin Baulig  <martin@@gnome.org>
Last Update: 2003-05-11
Rev: 0.5

* Purpose

This document captures all the good ideas people have had about
writing NUnit tests for the mono project. This document will be useful
for anyone who writes or maintains unit tests.

* Other resources
	- mcs/class/README has an explanation of the build process and
	  how it relates to the tests.
	- http://nunit.org is the place to find out about NUnit

* Getting Started

If you are new to writing NUnit tests, there is a template you may use
to help get started. The file is:

	mcs/class/doc/TemplateTest.cs

Save a copy of this file in the appropriate test subdirecty (see
below), and replace all the {text} markers with appropriate
code. Comments in the template are there to guide you. You should also
look at existing tests to see how other people have written them.
mcs/class/corlib/Test/System.Collections/CollectionBaseTest.cs is a
small one that might help.

The directory that will contain your new file depends on the
assembly/namespace of the class for which you are creating the
tests. Under mcs/class there is a directory for each assembly. In each
assembly there is a Test directory, e.g. mcs/class/corlib/Test. In the
Test directory there are sub-directories for each namespace in the
assembly, e.g. mcs/class/corlib/Test/Sytem. Put your new test file in
the appropriate sub-directory under Test for the class you are
testing.

Once all of that is done, you can do a 'make test' from the top mcs
directory.  Your test class will be automagically included in the
build and the tests will be run along with all the others.

* Tips

-- Provide an unique error message for Assertion.Assert ()

Include an unique message for each Assertion.Assert () so that when the assert
fails, it is trivial to locate the failing one. Otherwise, it may be
difficult to determine which part of the test is failing. A good way
to ensure unique messages is to use something like #A01, #A02 etc.

    Bad:

	Assertion.AssertEquals ("array match", compare[0], i1[0]);
	Assertion.AssertEquals ("array match", compare[1], i1[1]);
	Assertion.AssertEquals ("array match", compare[2], i1[2]);
	Assertion.AssertEquals ("array match", compare[3], i1[3]);

    Good:

	Assertion.AssertEquals ("#A01", compare[0], i1[0]);
	Assertion.AssertEquals ("#A02", compare[1], i1[1]);
	Assertion.AssertEquals ("#A03", compare[2], i1[2]);
	Assertion.AssertEquals ("#A04", compare[3], i1[3]);

Once you used such a number in an Assertion.Assert (), don't change it later on -
people might use it it identify the test in bug reports or in mailing
lists.

-- Use Assertion.AssertEquals () to compare things, not Assertion.Assert ().

Never compare two values with Assertion.Assert () - if the test fails, people
have no idea what went wrong while Assertion.AssertEquals () reports the failed
value. Also, make sure the second paramter is the expected value, and the third
parameter is the actual value.

    Bad:

        Assertion.Assert ("A01", myTicks[0] == t1.Ticks);

    Good:

        Assertion.AssertEquals ("A01", myTicks[0], t1.Ticks);

-- Namespace

Please keep the namespace within each test directory consistent. Of course 
you can use subnamespaces as you like - especially for subdirectories of 
your testsuite.

-- Test your test with the microsoft runtime

If possible, try to run your testsuite with the Microsoft runtime on
Windows and make sure all tests in it pass. This is especially
important if you're writing a totally new testcase - without this
check you can never be sure that your testcase contains no bugs ....

Don't worry if you're writing your test on Linux, other people can
test it for you on Windows.

Sometimes you may discover that a test doesn't show the expected
result when run with the Microsoft runtime - either because there is a
bug in their runtime or something is misleading or wrong in their
documentation. In this case, please put a detailed description of the
problem to mcs/class/doc/API-notes and do also report it to the list -
we'll forward this to the Microsoft people from time to time to help
them fix their documentation and runtime.