Unit Tests and Debug.Assert don't come usually work well together, specially if Debug.Assert pops out a dialog window that stops the execution of tests. MbUnit features a new test decorator that "digests" Debug.Assert calls and "translate" them for MbUnit. Simply tag the test that could trigger Debug.Assert with ExpectedDebugAssertAttribute.

[TestFixture]
public class ExpectedDebugAssertionAttributeTest
{
    [Test]
    [ExpectedDebugAssertion]
    public void ThrowAssertion()
    {
         Debug.Assert(false,"Testing assertion redirection");
     }
}

In the background, the decorator stores the listener in a temporary collection, emptis the Debug.Listeners collection and adds a simple listener that will throw in case of failure. When finishes, Debug.Listeners is rolledback.

The test method ThrowAssertion will fail if we do not trap the exception with ExpectedExceptionAttribute. In fact, in MbUnit, you can chain decorators:

[TestFixture]
public class ExpectedDebugAssertionAttributeTest
{
    [Test]
    [ExpectedException(typeof(AssertionException))] // this will trap the exceptoin
    [ExpectedDebugAssertion] // this will filter Debug.Assert to AssertionException
    public void ThrowAssertion()
    {
         Debug.Assert(false,"Testing assertion redirection");
     }
}

Attention, the order of declaration of decorators is important!

MbUnit features a new test decorator that asserts that the duration of the test is less than some values. DurationAttribute usage is quite intuitive as shown in the example below:

[TestFixture]
public class DurationAttributeTest
{
    [Test]
    [Duration(1,"This method succeeds (has 1 second to succeed")]
    public void EmptyMethod()
    {}

   [Test]
   [Duration(0,"This method will fail because it will  run in more that 0 seconds!")]
   public void EmptyMethodFails()
   {
      Console.WriteLine("Take your time");
   }
}

My blog is on (will be soon) the MSDN Belux site :)

http://www.microsoft.com/belux/fr/msdn/community/blogs.mspx

The TypeFixture lets you write a fixture for specific type and apply to a number of instance of this type.This fixture is particularly usefull for writing fixtures of interfaces and apply it to all the types that implement the interface.

In this tutorial, we are going to write a fixture for the IEnumerable interface (note that this interface is best tested with EnumerationFixture).

Fixture logic

The TypeFixture has the following execution logic:

1. (optional)Set-up the fixture, (SetUpAttribute),
2. Get an instance of the tested type provided by the user (ProviderAttribute),
3. Get instances of the tested type provided by a factory (ProviderFactoryAttribute),
4. Run test method with this instance as argument (TestAttribute)
5. (optional)Teardown fixture (TearDownAttribute)

Step 1: Creating the fixture

Create a new class EnumerableTest and tag it with TypeFixture. The TypeFixture attribute takes the tested type as parameter:

using System;
using System.Collections;
using MbUnit.Core.Framework;
using MbUnit.Framework;

[TypeFixture(typeof(IEnumerable))]
public EnumerableFixture
{
}

Step 2:  Create providers

MbUnit needs instance of the tested type to feed them into the different tests. Creating those instances is the job of tester. To do so, you can either

• write a method in the fixture, tagged with ProviderAttribute that returns an instance of the tested type,
• give the type of an instance factory that will be used to "produce" new instances
• or do both appraoch

We start by writing two methods that return respectively and enumerator on a empty list and non-empty list:

using System;
using System.Collections;
using MbUnit.Core.Framework;
using MbUnit.Framework;

[TypeFixture(typeof(IEnumerable))]
public EnumerableFixture
{
    [Provider(typeof(IEnumerable))]
    public ArrayList ProviderEmptyArrayList()
    {
        return new ArrayList();
    }

    [Provider(typeof(IEnumerable))]
    public ArrayList ProviderArrayList()
    {
        ArrayList list = new ArrayList();
        list.Add(0);
        list.Add(1);
        return list;
    }
}

The disadvantage of "hardcoding" methods in the fixture is that we cannot reuse the code for other fixture, for example, for the IList fixture. To avoid this problem you can wrap you "generation" methods in a class factory and use the ProviderFactory to tell MbUnit to use this factory:

using System;
using System.Collections;
using MbUnit.Core.Framework;
using MbUnit.Framework;

public class ArrayListFactory
{
    public ArrayList Empty
    {
        get
        {
            return new ArrayList();
        }
    }

    public ArrayList TwoElems
    {
        get
        {
            ArrayList list = new ArrayList();
            list.Add(0);
            list.Add(1);
            return list;
        }
    }
}

[TypeFixture(typeof(IEnumerator))]
[ProviderFactory(typeof(ArrayListFactory), typeof(IEnumerable))]
public EnumeratorFixture
{
}
That's much better because we can reuse the factory for other fixtures. Of course, you can attach an arbitrary number of factories and provider methods.
Step 3: Add some tests
Add the unit tests as usuals on the IEnumerable instance. These methods must take the tested type the tested type as argument.
Here we check that the enumerator throws if Current is called while the cursor if before the first element or past the last element:
...
[TypeFixture(typeof(IEnumerable))]
[ProviderFactory(typeof(ArrayListFactory), typeof(IEnumerable))]
public EnumerableFixture
{
    ...

    [Test]
    [ExpectedException(
         typeof(InvalidOperationException),
         "Current called while cursor is before the first element"
    )]
    public void CurrentCalledBeforeMoveNext(IEnumerable en)
    {
          IEnumerable er = en.GetEnumerator(); 
          en.Current;
    }

    [Test]
    [ExpectedException(
         typeof(InvalidOperationException),
         "Current called while cursor is past the last element"
    )]
    public void CurrentCalledBeforeMoveNext(IEnumerable en)
    {
          IEnumerable er = en.GetEnumerator(); 
          while(en.MoveNext());
          en.Current;
    }
}
Step 4: Compile and run
Compile and load in the GUI. MbUnit will scan the providers and create a fixture for each one of them.

This is a step-by-step tutorial to create your first MbUnit test fixture. (Note for NUnit, csUnit user: it is the same syntax).

Step 1: Create and set-up a project

• Create a library project (assembly) in your favorite .Net language (here it will be C#).
• Add the following references:
  • MbUnit.Core.dll
  • MbUnit.Framework.dll

Step 2: Create a TestFixture class

Create a new class MyFirstTest, and tag it with the attribute TestFixture. This will tell  MbUnit that this class contains tests to be executed.

using MbUnit.Core.Framework;
using MbUnit.Framework;

[TestFixture("This is my first test")]
public class MyFirstTest
{
}

Step 3: Add test methods

At this point, MyFirstTest does not have any test. Let's add a method that check that 1+1 = 2. To do so, we add the method OnePlusOneEqualTwo. In the test methods, you can use the assertion checks provided by the Assert static helper class:

[TestFixture("This is my first test")]
public class MyFirstTest
{
    [Test]
    public void OnePlusOneEqualTwo()
    {
         Assert.AreEqual(2, 1+1, "This is an error message");
    }
}

The signature of  OnePlusOneEqualTwo is important, it must be

public void MethodName(void);

Step 4: Running the tests

Once you have compiled the assembly, launch the MbUnit gui. Drag and drop the dll or use context menu to load the test assembly. The tree of available tests should instantly appear on the window. The GUI scans the loaded assembly for types tagged with TestFixture attribute and populates the tree. The namespace are reflected into nodes in the tree.

Hit run to launch the tests. When a test is run succesfully, the background of it's tree icon is turning green, this color propagated to the parents. However, if a test fails, the icon turns red and the parents are also turned red. 

To have informating about a failed test run, click on the node that failed and take a look at the exception. 

MbUnit now features several new assertions that touch different aspects of the applications. They provide new tool to easily add complicated assertions into your unit tests. All the new asssertions usually come under two version: the positive and negative "Not" version.

Includsion: Between and In (Assert class)

Two new assertions in Assert: Between and In which behaves as their SQL cousins:

int i=0;
Assert.Between(i, 1, 10);

int[] list = new int{1,2,3};
Assert.In(0, list);

String: StringAssert class

String is a very special type that totally deserves it's own assertions. Using Regular expression (System.Text.RegularExpression.Regex class), we can assert for complicated patterns. For example, checking for SQL injection in a string.

StringAssert.IsNotEmpty("");       // asserts that the string is not empty,
StringAssert.Like("hello", @"\w"); // asserts if it matches a regular expression
StringAssert.FullMatch("12-2344-21",@"\d{2}-\d{4}-\d{2}");// asserts that the match is full

Security: SecuAssert class

Performs some basic security based assertion: check that user is in role, check authentication, etc...

SecuAssert.WindowsIsInAdmin(); // asserts that is in administrator role
SecuAssert.IsAuthenticated(); // asserts that is authenticated
...

Reflection: RefleAssert class

Asserts that types can be assigned to each other, are instance of, or contains desired members:

Type parent;
Type child;

RefleAssert.IsAssignableFrom(parent, child); // asserts that parent is assignable from child
RefleAssert.HasMethod(typeof(String), "Substring", int, int);// asserts that System.String posseses Substring(int, int) method

Performance: PerfAssert class

Provides timing assertions. In the future it might also include memory related assertions:

CountDownTimer cdt = PerfAssert.Duration(10); // asserting that the duration to cdt.Close will last less that 10secs
...
cdt.Stop(); // throws if lastes more that 10 secs.

Data: DataAssert class

Data assertions compare DataSet, DataTable, DataRow and DataColumns content and schema:

DataSet ds;
DataSet ds2;

DataAssert.AreSchemaEqual(ds,ds2); // asserts that schemas of ds, ds2 are equal
DataAssert.AreDataEqual(ds,ds2);   // asserts that data in ds,ds2 are equal
DataAssert.AreEqual(ds,ds2);       // asserts that data and schema of ds,ds2 are equal

DataTable t,t2;
DataAssert(t,t2);

XML assertions: XmlAssert class

Xml comparaison is entirely done by the XmlUnit project from http://xmlunit.sourceforge.net . 

Formating error message

Assertion that accepts an error message support a "String.Format" like syntax: you can pass a format string and arguments:

string name="Marc";
Assert.AreEqual("John",Marc,"The name {0} does not match {1}", "John",name);

All suggestions for other assertions are welcome...

Consider the problem writing a unit test for the IList.Add method. At first sight this is trivial (as an example, we check that IList.Add supports null reference):

[TypeFixture(typeof(IList))]
public class ListTest
{
   [Test("List accepts null values")]
   public void AddNull(IList list)
   {
       list.Add(null);
       ...
   }
}

Now what about the Readonly and the FixedSize wrapper of ArrayList. They implement IList, but calling IList.Add on them will throw a NotSupportedException. Using the classic ExpectedExceptionAttribute technique, the only solution is to create a special fixture for read-only and fixed size wrappers, and that involves a lot of code duplication.

To tackle this problem, I have added ConditionalExceptionAttribute, which expects an exception if a predicate is true. The predicate is define by a method of the fixture that returns a boolean. This solution is simple and avoids the duplicate of unit tests when possible. Let's addapt the example with the new decorator:

[TypeFixture(typeof(IList))]
public class ListTest
{
   public bool IsReadOnly(IList list)
   {
       return list.ReadOnly;
   }

   [Test("List accepts null values")]
   [ConditionalException(typeof(NotSupportedException),"IsReadOnly")]
   public void AddNull(IList list)
   {
       list.Add(null);
       ...
   }
}

When the test will be executed, if list.ReadOnly is true, the framework will expect to receive a NotSupportedException, but if it is false, he will not check for exception. Note that the parameters of the predicate must match the parameters of the test method and it must return a bool.

Let's consider a simple fixture with a unit test that must throw. In this case, we use the ExpectedExceptionAttribute to tell the framework that the method should throw:

[TestFixture]
public class MyFixture
{
    [Test]
    [ExpectedException(typeof(Exception))]
    public void ThisMethodThrows()
    { ... }
}

ExpectedExceptionAttrribute is called a Decorator attribute because it decorates and alter the behavior of a test. Another commonly used decorator is IgnoreAttribute to ignore a test.

In MbUnit, other decorators are available. Here I describe two of those: RepeatAttribute and ThreadedRepeatAttribute. RepeatAttribute will make the execution of the test repeated the desired number of times in the same thread while ThreadedRepeatAttribute will launch simultaneously a desired number of threads that will execute the method. RepeatAttribute create a sequence of execution, ThreadedRepeatAttribute create a parallel execution.

    [Test]
    [Repeat(10)] // this will make the RepeatedTest executed 10 times
    public void RepeatedTest()
    { ... }


    [Test]
    [ThreadedRepeat(10)] // this will make the RepeatedTest executed in 10 concurent threads
    public void AmIThreadSake()
    { ... }

Decorators are chained internally in the order that they are defined, therefore declaration order is important if you want to combine them. For example, the two following test methods have different signification:

• RepeatAndThrow means: repeat test 10 times, test should throw at each execution
• ThrowWhileRepeating: repeat test 10 times, one of the test should throw during the repetition

    [Test]
    [Repeat(10)]
    [ExpectedException(typeof(Exception))]
    public void RepeatAndThrow()
    { ... }

    [Test]
    [ExpectedException(typeof(Exception))]
    [Repeat(10)]
    public void ThrowWhileRepeating()
    { ... }

NDoc is an extensible application: you can easily provide your own "documentation renderer", usually called documenter. I have used this feature to integrate NLiterate within NDoc and take advantage the built-in functions of the GUI. Here is the detailled procedure to create the NLiterate documenter.

1) Create the project:

Create an assembly project named NDoc.Documenter.*** where you replace *** with the name of your documenter. It is important to have your assembly named like this otherwise it will not be loaded by NDoc. Add the reference to NDoc.Core.dll.

2) Create the documenter config object:

The config class will be attached to the property grid control in the NDoc gui. This way we can easily define the different parameters of the documenter. The config class must implement IDocumenterConfig interface, however it is easier to inherit from BaseDocumenterConfig:

public class LiterateDocumenterConfig : BaseDocumenterConfig
{
    private string outputFile;
    public LiterateDocumenterConfig()
    :base("Literate")
    {
        this.OutputFile =String.Format(".{0}doc{0}compile-log.txt",Path.DirectorySeparatorChar);
    }

    public string OutputFile
    {
        get
        {
            return this.outputFile;
        }
        set
        {
            this.outputFile = value; 
            this.SetDirty();
        }
    }
}

Note that when OutputFile is modified, we notify NDoc using the protected method SetDirty.

3) Create the documenter class

The documenter class is the main class, it renders the documentation. This class must implement IDocumenter but again, it is easier to use the abstract base class BaseDocumenter:

public class LiterateDocumenter : BaseDocumenter
{ 
    public LiterateDocumenter()
    :base("Literate") // this is the name that will appear in the NDoc list box
    {
        this.Clear();
    }
    #region IDocumenter
    public override void Build(Project project)
    {
        // step one, load snippets
        loadSnippets(project);
        // step two, compile the snippets
        compileSnippets(project);
        // step three, create the report
        compileReport(project);
    }
    public override void Clear()
    {
        this.Config=new LiterateDocumenterConfig();
    }
    public override DocumenterDevelopmentStatus DevelopmentStatus 
    {  
        get
        {
            return DocumenterDevelopmentStatus.Alpha;
        }
    }
    public override string MainOutputFile
    {
        get
        {
            return ((LiterateDocumenterConfig)this.Config).OutputFile;
        }
    }
    #endregion
}

4) Adding progress notification

In the code above, we have not notified NDoc about the progress of the documentation rendering. This can be done through two functions: OnDocBuildingStep and OnDocBuildingProgress. For example:

    public override void Build(Project project)
    {
        // step one, load snippets
        OnDocBuildingStep(0, "Start loading snippets");
        loadSnippets(project);

        // step two, compile the snippets
        OnDocBuildingStep(40, "Start loading snippets");
        compileSnippets(project);
        // step three, create the report
        OnDocBuildingProgress(40); // increasing progress percentage of 40%
        compileReport(project);
    }

5) Copy the assembly

The last step is to copy your assembly into the NDoc bin folder. When launched, NDoc looks for assemblies named "NDoc.Documenter...." and loads by reflection that documenter they are containing. That's it.

I have added a new fixture, ProcessTestFixture, that implements the process testing presented by Marc Clifton in http://www.codeproject.com/csharp/autp3.asp.

A process is an ordered sequence of unit tests.  As long as one test passes, the next test is run. Due to the structure of MbUnit, this was straightforward to implement (I'll about this later in the blog). The syntax of the fixture is similar to Marc Clifton example but I have merge Test, Sequence(...) to TestSequence(...):

[ProcessTestFixture]
public MyProcess
{
    [TestSequence(1)]
    public void Hello()
    {...}
    [TestSequence(2)]
    public void World()
    {...}
    ...
}

The implementation of the fixture was done by the following steps:

1. Create TestSequenceAttribute, a simple class inherited from TestPatternAttribute that stores the order,

   [AttributeUsage(AttributeTargets.Method, AllowMultiple=false, Inherited=true)]
   public sealed class TestSequenceAttribute : TestPatternAttribute
   {
       private int order;
     
       public TestSequenceAttribute(int order)
         {
           this.order = order;
       }
   
       public int Order
       {
           get
           {
                   return this.order;
           }
           set
           {
               this.order = value;
           }
       }
   }

2. Create ProcessMethodRun, which implements IRun. This is where the "process" logic is created, more specifically in the Reflect method: since this method builds the tree of IRunInvoker instance, creating a process is just a matter of creating a sequence of MethodRunInvoker:

   public class ProcessMethodRun : IRun
   {
       ...
       public virtual void Reflect(
   			        RunInvokerTree tree, 
   			        RunInvokerVertex parent, 
           Type t
       )
       {
           // first gather all methods, with order.
           ArrayList methods = new ArrayList();
           foreach(MethodInfo mi in 
                   TypeHelper.GetAttributedMethods(t,this.AttributeType))
           {
               // get sequence attribute
               TestSequenceAttribute seq = 
                   (TestSequenceAttribute)TypeHelper.GetFirstCustomAttribute(mi,typeof(TestSequenceAttribute)); 
               methods.Add( new OrderedMethod(mi, seq.Order) ); 
           }
   
           // sort the methods
           QuickSorter sorter = new QuickSorter();
           sorter.Sort(methods);
   
           // populate execution tree.
           RunInvokerVertex child = parent;
           foreach(OrderedMethod om in methods)
           {
               IRunInvoker invoker = InstanceInvoker(om.Method);
               child = tree.AddChild(child,invoker);
           } 
       }
   
       protected IRunInvoker InstanceInvoker(MethodInfo mi)
       {
           IRunInvoker invoker = new MethodRunInvoker(this,mi);
           return DecoratorPatternAttribute.DecoreInvoker(mi,invoker);
       }
   
       #region OrderedMethod
       private class OrderedMethod : IComparable
       {
           private MethodInfo method;
           private int order;
      
           public  OrderedMethod(MethodInfo method, int order)
           {
               this.method = method;
               this.order = order;
           }
   
   
           public MethodInfo Method
           {
               get
               {
                   return this.method;
               }
           }
           public int Order
           {
               get
               {
                   return this.order; 
               }
           }
           public int CompareTo(OrderedMethod obj)
           {
               return this.order.CompareTo(obj.order);
           }
      
           int IComparable.CompareTo(Object obj)
           {
               return this.CompareTo((OrderedMethod)obj);
           }
       }
       #endregion
   	}

3. Create ProcessTestFixtureAttribute, a class that inherits from TestFixturePatternAttribute and defines the fixture execution pipe:

   public class ProcessTestFixtureAttribute : TestFixturePatternAttribute 
   {
       public override IRun GetRun()
       {
           SequenceRun runs = new SequenceRun();
      
           OptionalMethodRun setup = new OptionalMethodRun(typeof(SetUpAttribute),false);
           runs.Runs.Add( setup );
      
           ProcessMethodRun test = new ProcessMethodRun(typeof(TestSequenceAttribute));
           runs.Runs.Add(test);
      
           OptionalMethodRun tearDown = new OptionalMethodRun(typeof(TearDownAttribute),false);
           runs.Runs.Add(tearDown);
      
           return runs;      
       }
   }

Et voilà!