-* Testing
+* Testing
+
+ Daily <a href="http://www.go-mono.com/tests/index.php">test</a> results.
+
+ Testing is an important part of the Mono project: every one of its
+ three major components has a test suite tailored for its needs. This
+ is very helpful, because in the course of developing the software it
+ is very common to introduce bugs in existing code. A test suite
+ helps us fix the bugs as soon as they are introduced.
+
+ There are various kinds of tests in Mono:
+ <ul>
+ <li><a href="#unit"><b>Class Library Unit
+ Tests:</b></a> These are used to test the class
+ libraries.
+
+ <li><a href="#compiler"><b>Compiler tests</b></a>: Both
+ tests that should pass and tests that should fail are included.
+
+ <li><a href="#runtime"><b>Runtime tests</b></a>: Tests for
+ the virtual machine.
+
+ <li><a href="#aspnet"><b>ASP.NET tests</b></a>: ASP.NET tests.
+
+ <li><a href="#ws"><b>Web Services tests</b></a>: Web Services
+ client/server tests.
+ </ul>
+
+<a name="unit"></a>
+* Class Library Tests
All classes in Mono libraries should have comprehensive unit test
- suites to go with them. Unit testing is a software engineering
- methodology that makes it easier to build correct code. Every
+ suites to go with them. Unit testing is a software engineering
+ methodology that makes it easier to build correct code. Every
method in every class should have a set of tests to verify
- they work correctly. Mono also needs a testing framework to
- make it easy to write and run lots of tests. <a
- href="http://www.junit.org/">JUnit</a>
- and its spinoffs (collectively, <a
- href="http://www.xprogramming.com/software.htm">XUnit</a>)
- are a well designed framework for building unit
- tests. Implementing JUnit in Mono is a first priority.
+ that they work correctly. Mono also needs a testing framework
+ to make it easy to write and run lots of tests.
+
+ In some classes, we might also provide standalone tests because of
+ some reasons such as too huge testcases, another downloading and so on.
+ (For example, managed XSLT has standalone test which downloads and
+ expands some megabytes of OASIS test suite.)
+
+ Here I list them up as long as I know. If you are going to add another
+ standalone tests, please add one line here. It is also recommended that
+ you add some notes on how to build and run tests.
+
+ <ul>
+
+ * Mono.Data/test/
+ * System.Data/Test, and some individual ADO.NET libraries:
+ there are some standalone tests. See the bottom of <a href="ado-net.html">
+ ADO.NET page</a> for detail.
+ * System.Web/Test/TestMonoWeb : see README
+ * System.Web.Services/Test/standalone : see README
+ * System.Windows.Forms/SWFTest/
+ * System.XML/Test/System.Xml/standalone_tests : see README
+ * System.XML/Test/System.Xml.Schema/standalone_tests : see README
+ * System.XML/System.Xml.Serialization/standalone_tests/
+ * System.XML/Test/System.Xml.Xsl/standalone_tests : see README
+ * Commons.Xml.Relaxng/Test/standalone_tests : see README
+
+ </ul>
+
+** Getting started
+
+ If you are new to writing NUnit tests, there is a template you may use
+ to help get started. The file is:
+
+ <b>mcs/class/doc/TemplateTest.cs</b>
+
+ Save a copy of this file in the appropriate test subdirecty
+ (see below), and replace all the {text} markers with
+ appropriate code. Comments in the template are there to guide
+ you. You should also look at existing tests to see how other
+ people have written them.
+ mcs/class/corlib/Test/System.Collections/CollectionBaseTest.cs
+ is a small one that might help.
+
+ The directory that will contain your new file depends on the
+ assembly/namespace of the class for which you are creating the
+ tests. Under mcs/class there is a directory for each
+ assembly. In each assembly there is a Test directory,
+ e.g. mcs/class/corlib/Test. In the Test directory there are
+ sub-directories for each namespace in the assembly,
+ e.g. mcs/class/corlib/Test/Sytem. Put your new test file in
+ the appropriate sub-directory under Test for the class you are
+ testing.
+
+ Once all of that is done, you can do a 'make test' from the top mcs
+ directory. Your test class needs also to be listed in the
+ .sources file at the top of the Test directory.
+
+* Tips on writing Unit tests.
+
+ You should look at the <a href="http://nunit.org">NUnit documentation</a>,
+ as it is a fantastic product, and includes fantastic documentation,
+ but here are some tips for those of you who are already reading
+ this web page.
+
+
+** Provide an unique error message for Assert()
+
+ Include an unique message for each Assert() so that when the assert
+ fails, it is trivial to locate it in the source. Otherwise, it may be
+ difficult to determine which part of the test is failing. A good way
+ to ensure unique messages is to use something like #A01, #A02 etc.
+
+ Ok:
+ <pre>
+
+ AssertEquals("array match", compare[0], i1[0]);
+ AssertEquals("array match", compare[1], i1[1]);
+ AssertEquals("array match", compare[2], i1[2]);
+ AssertEquals("array match", compare[3], i1[3]);
+ </pre>
+
+ Excellent:
+ <pre>
+ AssertEquals("#A01", compare[0], i1[0]);
+ AssertEquals("#A02", compare[1], i1[1]);
+ AssertEquals("#A03", compare[2], i1[2]);
+ AssertEquals("#A04", compare[3], i1[3]);
+ </pre>
+
+ Once you used such a number in an Assert(), don't change it later on -
+ people might use it it identify the test in bug reports or in mailing
+ lists.
+
+** Use AssertEquals() to compare things, not Assert().
+
+ Do not compare two values with Assert() - if the test fails,
+ people have no idea what went wrong while AssertEquals()
+ reports the failed value.
+
+ Ok:
+ <pre>
+ Assert ("A01", myTicks[0] == t1.Ticks);
+ </pre>
+
+ Excellent:
+ <pre>
+ AssertEquals ("A01", myTicks[0], t1.Ticks);
+ </pre>
+
+** Test your test with the Microsoft runtime
+
+ If possible, try to run your testsuite with the Microsoft runtime on
+ .NET on Windows and make sure all tests in it pass. This is especially
+ important if you're writing a totally new testcase - without this
+ check you can never be sure that your testcase contains no bugs ....
+
+ Don't worry if you're writing your test on Linux, other people can
+ test it for you on Windows.
+
+ Sometimes you may discover that a test doesn't show the expected
+ result when run with the Microsoft runtime - either because there is a
+ bug in their runtime or something is misleading or wrong in their
+ documentation. In this case, please put a detailed description of the
+ problem to mcs/class/doc/API-notes and do also report it to the
+ <a href="mailing-lists.html">mailing list</a> - we'll forward this to the
+ Microsoft people from time to time to help them fix their documentation
+ and runtime.
+
+** Unit tests.
Why do unit testing? It becomes simple to run automated tests
for the whole library. Unit tests are a safety net - you can
change part of the code and verify that you haven't broken
anything. Ideally, tests are written before the actual library
code itself. And every time a bug is discovered, a test should
- be written to demonstrate the bug, fix the bug, and then if
- you ever reintroduce the bug you will know immediately. For
+ be written to demonstrate the bug and its fix. Then, if
+ you ever reintroduce the bug, you will know immediately. For
more info, read <a
href="http://junit.sourceforge.net/doc/testinfected/testing.htm">
JUnit Test Infected: Programmers Love Writing Tests</a>.
+
+
+** Getting Started
+
+ We welcome all contributions to the Class Libary Test Suite.
+
+ There is information to help you get started in CVS at
+ mcs/class/doc/NUnitGuidelines. Once you have written your test, please
+ post it to <a href="mailing-lists.html">mono-list</a>.
+
+ Someone will make sure to add the file or apply the patch as
+ appropriate. If you plan to be an on-going contributor and
+ would like to get cvs account, email <a href="mailto:miguel@ximian.com">miguel</a>.
+
+ Normally, after you send a couple of well-written new files
+ and/or patches to the list, you will be given cvs access.
+
+<a name="compiler"></a>
+* Compiler tests
+
+ Mono ships with three compilers: C#, VB.NET and JScript. The
+ tests are ran by running the makefile target `make
+ run-test-local' in the appropriate directory.
+
+ The C# compilation tests live in mcs/tests, and the C# error
+ tests live in mcs/errors.
+
+ The VB.NET compilation tests live in mcs/btests.
+
+<a name="runtime"></a>
+* Runtime Tests
+
+ These tests verify the virtual machine, to run these tests, do:
+
+<pre>
+ cd mono/mono/tests
+ make test
+</pre>
+
+<a name="aspnet"></a>
+* ASP.NET tests
+
+ XSP, the Mono ASP.NET server has tests for ASP.NET pages. It uses
+ <a href="http://nunitasp.sourceforge.net">NUnitAsp</a>. Right now
+ it only has standalone tests, ie., tests that do not need their own
+ global.asax or web.config files.
+
+ If you want to run them, get the xsp CVS module and install it. Then:
+<pre>
+ cd xsp/nunit-tests
+ make
+ cd standalone
+ xsp
+</pre>
+
+ And from another terminal:
+<pre>
+ cd xsp/nunit-tests/standalone
+ nunit-console standalone-tests.dll
+</pre>
+
+<a name="ws"></a>
+* Web Services tests
+
+ The Test directory for the System.Web.Services assembly contains a
+ standalone test suite for testing web services. It tests:
+
+ <ul>
+ <li>Proxy generation using the wsdl tool</li>
+ <li>Access to web services using the generated client proxies</li>
+ <li>Execution of web services in the server</li>
+ </ul>
+
+ This suite not only tests web services running on XSP, but it can also test
+ services running on other platforms and that are available in internet. This
+ will help track down interoperability issues.
+
+ To build the test suite, just run:
+
+<pre>
+ cd mcs/class/System.Web.Services/Test/standalone
+ xsp --root server
+</pre>
+
+ And from another terminal:
+<pre>
+ cd mcs/class/System.Web.Services/Test/standalone
+ make
+ nunit-console testclient.dll
+</pre>
+
+ This will download the wsdl documents, generate the proxies, build a dll with
+ the proxies, and build the nunit tests. Then you can use nunit-console or
+ gnunit to run the tests (the nunit dll is testclient.dll).
+
+ Read the README file in mcs/class/System.Web.Services/Test/standalone for
+ more info.
projects / mono.git / blobdiff