MbUnit now features several new assertions that touch different aspects of the applications. They provide new tool to easily add complicated assertions into your unit tests. All the new asssertions usually come under two version: the positive and negative "Not" version.

Includsion: Between and In (Assert class)

Two new assertions in Assert: Between and In which behaves as their SQL cousins:

int i=0;
Assert.Between(i, 1, 10);

int[] list = new int{1,2,3};
Assert.In(0, list);

String: StringAssert class

String is a very special type that totally deserves it's own assertions. Using Regular expression (System.Text.RegularExpression.Regex class), we can assert for complicated patterns. For example, checking for SQL injection in a string.

StringAssert.IsNotEmpty("");       // asserts that the string is not empty,
StringAssert.Like("hello", @"\w"); // asserts if it matches a regular expression
StringAssert.FullMatch("12-2344-21",@"\d{2}-\d{4}-\d{2}");// asserts that the match is full

Security: SecuAssert class

Performs some basic security based assertion: check that user is in role, check authentication, etc...

SecuAssert.WindowsIsInAdmin(); // asserts that is in administrator role
SecuAssert.IsAuthenticated(); // asserts that is authenticated
...

Reflection: RefleAssert class

Asserts that types can be assigned to each other, are instance of, or contains desired members:

Type parent;
Type child;

RefleAssert.IsAssignableFrom(parent, child); // asserts that parent is assignable from child
RefleAssert.HasMethod(typeof(String), "Substring", int, int);// asserts that System.String posseses Substring(int, int) method

Performance: PerfAssert class

Provides timing assertions. In the future it might also include memory related assertions:

CountDownTimer cdt = PerfAssert.Duration(10); // asserting that the duration to cdt.Close will last less that 10secs
...
cdt.Stop(); // throws if lastes more that 10 secs.

Data: DataAssert class

Data assertions compare DataSet, DataTable, DataRow and DataColumns content and schema:

DataSet ds;
DataSet ds2;

DataAssert.AreSchemaEqual(ds,ds2); // asserts that schemas of ds, ds2 are equal
DataAssert.AreDataEqual(ds,ds2);   // asserts that data in ds,ds2 are equal
DataAssert.AreEqual(ds,ds2);       // asserts that data and schema of ds,ds2 are equal

DataTable t,t2;
DataAssert(t,t2);

XML assertions: XmlAssert class

Xml comparaison is entirely done by the XmlUnit project from http://xmlunit.sourceforge.net . 

Formating error message

Assertion that accepts an error message support a "String.Format" like syntax: you can pass a format string and arguments:

string name="Marc";
Assert.AreEqual("John",Marc,"The name {0} does not match {1}", "John",name);

All suggestions for other assertions are welcome...

Consider the problem writing a unit test for the IList.Add method. At first sight this is trivial (as an example, we check that IList.Add supports null reference):

[TypeFixture(typeof(IList))]
public class ListTest
{
   [Test("List accepts null values")]
   public void AddNull(IList list)
   {
       list.Add(null);
       ...
   }
}

Now what about the Readonly and the FixedSize wrapper of ArrayList. They implement IList, but calling IList.Add on them will throw a NotSupportedException. Using the classic ExpectedExceptionAttribute technique, the only solution is to create a special fixture for read-only and fixed size wrappers, and that involves a lot of code duplication.

To tackle this problem, I have added ConditionalExceptionAttribute, which expects an exception if a predicate is true. The predicate is define by a method of the fixture that returns a boolean. This solution is simple and avoids the duplicate of unit tests when possible. Let's addapt the example with the new decorator:

[TypeFixture(typeof(IList))]
public class ListTest
{
   public bool IsReadOnly(IList list)
   {
       return list.ReadOnly;
   }

   [Test("List accepts null values")]
   [ConditionalException(typeof(NotSupportedException),"IsReadOnly")]
   public void AddNull(IList list)
   {
       list.Add(null);
       ...
   }
}

When the test will be executed, if list.ReadOnly is true, the framework will expect to receive a NotSupportedException, but if it is false, he will not check for exception. Note that the parameters of the predicate must match the parameters of the test method and it must return a bool.

Let's consider a simple fixture with a unit test that must throw. In this case, we use the ExpectedExceptionAttribute to tell the framework that the method should throw:

[TestFixture]
public class MyFixture
{
    [Test]
    [ExpectedException(typeof(Exception))]
    public void ThisMethodThrows()
    { ... }
}

ExpectedExceptionAttrribute is called a Decorator attribute because it decorates and alter the behavior of a test. Another commonly used decorator is IgnoreAttribute to ignore a test.

In MbUnit, other decorators are available. Here I describe two of those: RepeatAttribute and ThreadedRepeatAttribute. RepeatAttribute will make the execution of the test repeated the desired number of times in the same thread while ThreadedRepeatAttribute will launch simultaneously a desired number of threads that will execute the method. RepeatAttribute create a sequence of execution, ThreadedRepeatAttribute create a parallel execution.

    [Test]
    [Repeat(10)] // this will make the RepeatedTest executed 10 times
    public void RepeatedTest()
    { ... }


    [Test]
    [ThreadedRepeat(10)] // this will make the RepeatedTest executed in 10 concurent threads
    public void AmIThreadSake()
    { ... }

Decorators are chained internally in the order that they are defined, therefore declaration order is important if you want to combine them. For example, the two following test methods have different signification:

• RepeatAndThrow means: repeat test 10 times, test should throw at each execution
• ThrowWhileRepeating: repeat test 10 times, one of the test should throw during the repetition

    [Test]
    [Repeat(10)]
    [ExpectedException(typeof(Exception))]
    public void RepeatAndThrow()
    { ... }

    [Test]
    [ExpectedException(typeof(Exception))]
    [Repeat(10)]
    public void ThrowWhileRepeating()
    { ... }

NDoc is an extensible application: you can easily provide your own "documentation renderer", usually called documenter. I have used this feature to integrate NLiterate within NDoc and take advantage the built-in functions of the GUI. Here is the detailled procedure to create the NLiterate documenter.

1) Create the project:

Create an assembly project named NDoc.Documenter.*** where you replace *** with the name of your documenter. It is important to have your assembly named like this otherwise it will not be loaded by NDoc. Add the reference to NDoc.Core.dll.

2) Create the documenter config object:

The config class will be attached to the property grid control in the NDoc gui. This way we can easily define the different parameters of the documenter. The config class must implement IDocumenterConfig interface, however it is easier to inherit from BaseDocumenterConfig:

public class LiterateDocumenterConfig : BaseDocumenterConfig
{
    private string outputFile;
    public LiterateDocumenterConfig()
    :base("Literate")
    {
        this.OutputFile =String.Format(".{0}doc{0}compile-log.txt",Path.DirectorySeparatorChar);
    }

    public string OutputFile
    {
        get
        {
            return this.outputFile;
        }
        set
        {
            this.outputFile = value; 
            this.SetDirty();
        }
    }
}

Note that when OutputFile is modified, we notify NDoc using the protected method SetDirty.

3) Create the documenter class

The documenter class is the main class, it renders the documentation. This class must implement IDocumenter but again, it is easier to use the abstract base class BaseDocumenter:

public class LiterateDocumenter : BaseDocumenter
{ 
    public LiterateDocumenter()
    :base("Literate") // this is the name that will appear in the NDoc list box
    {
        this.Clear();
    }
    #region IDocumenter
    public override void Build(Project project)
    {
        // step one, load snippets
        loadSnippets(project);
        // step two, compile the snippets
        compileSnippets(project);
        // step three, create the report
        compileReport(project);
    }
    public override void Clear()
    {
        this.Config=new LiterateDocumenterConfig();
    }
    public override DocumenterDevelopmentStatus DevelopmentStatus 
    {  
        get
        {
            return DocumenterDevelopmentStatus.Alpha;
        }
    }
    public override string MainOutputFile
    {
        get
        {
            return ((LiterateDocumenterConfig)this.Config).OutputFile;
        }
    }
    #endregion
}

4) Adding progress notification

In the code above, we have not notified NDoc about the progress of the documentation rendering. This can be done through two functions: OnDocBuildingStep and OnDocBuildingProgress. For example:

    public override void Build(Project project)
    {
        // step one, load snippets
        OnDocBuildingStep(0, "Start loading snippets");
        loadSnippets(project);

        // step two, compile the snippets
        OnDocBuildingStep(40, "Start loading snippets");
        compileSnippets(project);
        // step three, create the report
        OnDocBuildingProgress(40); // increasing progress percentage of 40%
        compileReport(project);
    }

5) Copy the assembly

The last step is to copy your assembly into the NDoc bin folder. When launched, NDoc looks for assemblies named "NDoc.Documenter...." and loads by reflection that documenter they are containing. That's it.

I have added a new fixture, ProcessTestFixture, that implements the process testing presented by Marc Clifton in http://www.codeproject.com/csharp/autp3.asp.

A process is an ordered sequence of unit tests.  As long as one test passes, the next test is run. Due to the structure of MbUnit, this was straightforward to implement (I'll about this later in the blog). The syntax of the fixture is similar to Marc Clifton example but I have merge Test, Sequence(...) to TestSequence(...):

[ProcessTestFixture]
public MyProcess
{
    [TestSequence(1)]
    public void Hello()
    {...}
    [TestSequence(2)]
    public void World()
    {...}
    ...
}

The implementation of the fixture was done by the following steps:

1. Create TestSequenceAttribute, a simple class inherited from TestPatternAttribute that stores the order,

   [AttributeUsage(AttributeTargets.Method, AllowMultiple=false, Inherited=true)]
   public sealed class TestSequenceAttribute : TestPatternAttribute
   {
       private int order;
     
       public TestSequenceAttribute(int order)
         {
           this.order = order;
       }
   
       public int Order
       {
           get
           {
                   return this.order;
           }
           set
           {
               this.order = value;
           }
       }
   }

2. Create ProcessMethodRun, which implements IRun. This is where the "process" logic is created, more specifically in the Reflect method: since this method builds the tree of IRunInvoker instance, creating a process is just a matter of creating a sequence of MethodRunInvoker:

   public class ProcessMethodRun : IRun
   {
       ...
       public virtual void Reflect(
   			        RunInvokerTree tree, 
   			        RunInvokerVertex parent, 
           Type t
       )
       {
           // first gather all methods, with order.
           ArrayList methods = new ArrayList();
           foreach(MethodInfo mi in 
                   TypeHelper.GetAttributedMethods(t,this.AttributeType))
           {
               // get sequence attribute
               TestSequenceAttribute seq = 
                   (TestSequenceAttribute)TypeHelper.GetFirstCustomAttribute(mi,typeof(TestSequenceAttribute)); 
               methods.Add( new OrderedMethod(mi, seq.Order) ); 
           }
   
           // sort the methods
           QuickSorter sorter = new QuickSorter();
           sorter.Sort(methods);
   
           // populate execution tree.
           RunInvokerVertex child = parent;
           foreach(OrderedMethod om in methods)
           {
               IRunInvoker invoker = InstanceInvoker(om.Method);
               child = tree.AddChild(child,invoker);
           } 
       }
   
       protected IRunInvoker InstanceInvoker(MethodInfo mi)
       {
           IRunInvoker invoker = new MethodRunInvoker(this,mi);
           return DecoratorPatternAttribute.DecoreInvoker(mi,invoker);
       }
   
       #region OrderedMethod
       private class OrderedMethod : IComparable
       {
           private MethodInfo method;
           private int order;
      
           public  OrderedMethod(MethodInfo method, int order)
           {
               this.method = method;
               this.order = order;
           }
   
   
           public MethodInfo Method
           {
               get
               {
                   return this.method;
               }
           }
           public int Order
           {
               get
               {
                   return this.order; 
               }
           }
           public int CompareTo(OrderedMethod obj)
           {
               return this.order.CompareTo(obj.order);
           }
      
           int IComparable.CompareTo(Object obj)
           {
               return this.CompareTo((OrderedMethod)obj);
           }
       }
       #endregion
   	}

3. Create ProcessTestFixtureAttribute, a class that inherits from TestFixturePatternAttribute and defines the fixture execution pipe:

   public class ProcessTestFixtureAttribute : TestFixturePatternAttribute 
   {
       public override IRun GetRun()
       {
           SequenceRun runs = new SequenceRun();
      
           OptionalMethodRun setup = new OptionalMethodRun(typeof(SetUpAttribute),false);
           runs.Runs.Add( setup );
      
           ProcessMethodRun test = new ProcessMethodRun(typeof(TestSequenceAttribute));
           runs.Runs.Add(test);
      
           OptionalMethodRun tearDown = new OptionalMethodRun(typeof(TearDownAttribute),false);
           runs.Runs.Add(tearDown);
      
           return runs;      
       }
   }

Et voilà!

It is very easy to compute the code coverage of the Unit tests using MbUnit and NCover.

1. Install the latest version of NCover (from gotdotnet: http://www.gotdotnet.com/Community/Workspaces/Workspace.aspx?id=3122ee1a-46e7-48a5-857e-aad6739ef6b9 )
2. Create a test assembly and compile it in debug mode. NCover needs the symbol files to work (.mdb)
3. Launch NCover as follows:

   NCover.Console.exe /c "MbUnit.Cons.exe" "MyTestAssembly.dll" /a "TheAssemblyToComputeTheCoverageOn"

   where MyTestAssembly.dll is the unit test assembly, TheAssemblyToComputeTheCoverageOn is the name of the assembly to compute the code coverage on (without the .dll extension)

4. That' it.

You can launch multiple test assembly by providing multiple assembly names. And you can monitor multiple assembly, by providing a ;-separated list of assemblies.

I will be presenting MbUnit at the end of May on a BENUG meeting. BENUG stands for Belgian User Group, http://www.benug.be .

NDoc is truly an amazing library.

You can easily define your own documentation renderer (Documenter) class by implementing the NDoc.Core.IDocumenter interface (inheriting from BaseDocumenter will help). Once this is done, put the assembly ( named NDoc.Documenter.YourRenderer.dll) into the directory where NDocGUI.exe is and it will be loaded by Reflection where the GUI opens.

Since NLiterate is one way of looking at documentation, it deserved it's own documenter. The task was surprinsgly easy... maybe this could be my 2cents for NDoc :)

I have published a new article on CodeProject about NLiterate ( http://www.codeproject.com/useritems/nliterate.asp )... don't forget to vote :)

Literate programming ( http://www.literateprogramming.com  ) is a way of describing algorithms and programs invented by the the mythical D. Knuth (famous CWEB application). The nice thing about CWEB is that both code and documentation live in the same document. To compile the code, the tool reassembles the code snippets and compiles it. Therefore, you can verify that all the examples in your documentation are compilable. Sadly, the (excellent) C# documentation does not support literate programming... well now it does.

NLiterate is a small application that parses the XML documentation file, reassembles code section and compiles them. This gives you an automated way of testing all your documentation examples. I'll be posting the source soon.