X-Git-Url: http://wien.tomnetworks.com/gitweb/?a=blobdiff_plain;f=web%2Ftesting;h=ab60d7fe4d3f88aef1074b23f49c955a094f9b45;hb=6d244cc6a029eb048a31840c7e89ea477a08c80b;hp=f97902ec306890f7129fb36b067a56d55b57301a;hpb=3d41c6a0248cc4e9d6d6f00e70af516b4b6521c9;p=mono.git diff --git a/web/testing b/web/testing index f97902ec306..ab60d7fe4d3 100644 --- a/web/testing +++ b/web/testing @@ -1,21 +1,168 @@ * Testing + Daily test results. + Testing is an important part of the Mono project: every one of its three major components has a test suite tailored for its needs. This is very helpful, because in the course of developing the software it is very common to introduce bugs in existing code. A test suite helps us fix the bugs as soon as they are introduced. -** Class Library Tests + There are various kinds of tests in Mono: + + + +* Class Library Tests All classes in Mono libraries should have comprehensive unit test - suites to go with them. Unit testing is a software engineering - methodology that makes it easier to build correct code. Every + suites to go with them. Unit testing is a software engineering + methodology that makes it easier to build correct code. Every method in every class should have a set of tests to verify - that they work correctly. Mono also needs a testing framework + that they work correctly. Mono also needs a testing framework to make it easy to write and run lots of tests. - Try NUnit + In some classes, we might also provide standalone tests because of + some reasons such as too huge testcases, another downloading and so on. + (For example, managed XSLT has standalone test which downloads and + expands some megabytes of OASIS test suite.) + + Here I list them up as long as I know. If you are going to add another + standalone tests, please add one line here. It is also recommended that + you add some notes on how to build and run tests. + + + +** Getting started + + If you are new to writing NUnit tests, there is a template you may use + to help get started. The file is: + + mcs/class/doc/TemplateTest.cs + + Save a copy of this file in the appropriate test subdirecty + (see below), and replace all the {text} markers with + appropriate code. Comments in the template are there to guide + you. You should also look at existing tests to see how other + people have written them. + mcs/class/corlib/Test/System.Collections/CollectionBaseTest.cs + is a small one that might help. + + The directory that will contain your new file depends on the + assembly/namespace of the class for which you are creating the + tests. Under mcs/class there is a directory for each + assembly. In each assembly there is a Test directory, + e.g. mcs/class/corlib/Test. In the Test directory there are + sub-directories for each namespace in the assembly, + e.g. mcs/class/corlib/Test/Sytem. Put your new test file in + the appropriate sub-directory under Test for the class you are + testing. + + Once all of that is done, you can do a 'make test' from the top mcs + directory. Your test class needs also to be listed in the + .sources file at the top of the Test directory. + +* Tips on writing Unit tests. + + You should look at the NUnit documentation, + as it is a fantastic product, and includes fantastic documentation, + but here are some tips for those of you who are already reading + this web page. + + +** Provide an unique error message for Assert() + + Include an unique message for each Assert() so that when the assert + fails, it is trivial to locate it in the source. Otherwise, it may be + difficult to determine which part of the test is failing. A good way + to ensure unique messages is to use something like #A01, #A02 etc. + + Ok: + 
+	
+		AssertEquals("array match", compare[0], i1[0]);
+		AssertEquals("array match", compare[1], i1[1]);
+		AssertEquals("array match", compare[2], i1[2]);
+		AssertEquals("array match", compare[3], i1[3]);
+
+ + Excellent: + 
+		AssertEquals("#A01", compare[0], i1[0]);
+		AssertEquals("#A02", compare[1], i1[1]);
+		AssertEquals("#A03", compare[2], i1[2]);
+		AssertEquals("#A04", compare[3], i1[3]);
+
+ + Once you used such a number in an Assert(), don't change it later on - + people might use it it identify the test in bug reports or in mailing + lists. + +** Use AssertEquals() to compare things, not Assert(). + + Do not compare two values with Assert() - if the test fails, + people have no idea what went wrong while AssertEquals() + reports the failed value. + + Ok: + 
+	        Assert ("A01", myTicks[0] == t1.Ticks);
+
+ + Excellent: + 
+	        AssertEquals ("A01", myTicks[0], t1.Ticks);
+
+ +** Test your test with the Microsoft runtime + + If possible, try to run your testsuite with the Microsoft runtime on + .NET on Windows and make sure all tests in it pass. This is especially + important if you're writing a totally new testcase - without this + check you can never be sure that your testcase contains no bugs .... + + Don't worry if you're writing your test on Linux, other people can + test it for you on Windows. + + Sometimes you may discover that a test doesn't show the expected + result when run with the Microsoft runtime - either because there is a + bug in their runtime or something is misleading or wrong in their + documentation. In this case, please put a detailed description of the + problem to mcs/class/doc/API-notes and do also report it to the + mailing list - we'll forward this to the + Microsoft people from time to time to help them fix their documentation + and runtime. + +** Unit tests. Why do unit testing? It becomes simple to run automated tests for the whole library. Unit tests are a safety net - you can @@ -44,3 +191,83 @@ Normally, after you send a couple of well-written new files and/or patches to the list, you will be given cvs access. + +* Compiler tests + + Mono ships with three compilers: C#, VB.NET and JScript. The + tests are ran by running the makefile target `make + run-test-local' in the appropriate directory. + + The C# compilation tests live in mcs/tests, and the C# error + tests live in mcs/errors. + + The VB.NET compilation tests live in mcs/btests. + + +* Runtime Tests + + These tests verify the virtual machine, to run these tests, do: + +
+	cd mono/mono/tests
+	make test
+
+ + +* ASP.NET tests + + XSP, the Mono ASP.NET server has tests for ASP.NET pages. It uses + NUnitAsp. Right now + it only has standalone tests, ie., tests that do not need their own + global.asax or web.config files. + + If you want to run them, get the xsp CVS module and install it. Then: +
+	cd xsp/nunit-tests
+	make
+	cd standalone
+	xsp
+
+ + And from another terminal: +
+	cd xsp/nunit-tests/standalone
+	nunit-console standalone-tests.dll
+
+ + +* Web Services tests + + The Test directory for the System.Web.Services assembly contains a + standalone test suite for testing web services. It tests: + + + + This suite not only tests web services running on XSP, but it can also test + services running on other platforms and that are available in internet. This + will help track down interoperability issues. + + To build the test suite, just run: + +
+	cd mcs/class/System.Web.Services/Test/standalone
+	xsp --root server
+
+ + And from another terminal: +
+	cd mcs/class/System.Web.Services/Test/standalone
+	make
+	nunit-console testclient.dll
+
+ + This will download the wsdl documents, generate the proxies, build a dll with + the proxies, and build the nunit tests. Then you can use nunit-console or + gnunit to run the tests (the nunit dll is testclient.dll). + + Read the README file in mcs/class/System.Web.Services/Test/standalone for + more info.