The last few days have been very exciting for MbUnit. I have been working (remotely) with Jamie Cansdale, creator of NUnitAddIn, to create a Visual Studio Add-in for MbUnit. It turned out to be surpringly simple, thanks to the extensible architecture of NUnitAddIn and the expertise of Jamie.

A quick NUnitAddIn introduction

NUnitAddIn is not just an Add-in for NUnit as it's name seems to tell. It is far more than that. It is a extensible framework to build Add-ins. All the words are important here: extensible, because it is designed to accept any type of Add-in (at least test runners) and framework because it takes care of all the complicated/technical task of launching processes, attaching debuggers, etc. In fact, writing an Add-in with NUnitAddIn is as simple as implementing an interface!

Add-in How-to

I will give here a detailled how-to on the Add-in creation. This is the summary of few hours of coding and dozens of MSN messages with Jamie Cansdale (very active support!). Now let's go for the fun (I will assume you have NUnitAddIn installed in c:\Program Files\NUnitAddIn)

Setup the project

1. Create a new Assembly project and name it as you like. In this example, it will be named MbUnit.AddIn
2. Add the following references
   • NUnitAddIn.TestRunner.dll
   • NUnitAddIn.TestRunner.Framework.dll
3. Change to ouput directory to the NUnitAddIn directory: Projet -> Properties -> Common Properties -> Output Path -> Select NUnitAddIn directory
4. Make the assembly strongly named

Setup NUnitAddIn

You need to edit NUnitAddIn.config in the NUnitAddIn directory (do not edit NUnitAddIn.exe.config). The file looks like this:

<?xml version="1.0"?>
<configuration>
  <nunitaddin>
    <frameworktestrunners>
      <testrunner name="NUnit"
                     typeName="NUnitAddin.NUnit.TestRunner.SimpleNUnitTestRunner" 
                     assemblyPath="NUnitAddin.NUnit.dll"  />
      ...
    </frameworktestrunners>
  </nunitaddin>
</configuration>

Add your Add-in on top of the "food chain":

<?xml version="1.0"?>
<configuration>
  <nunitaddin>
    <frameworktestrunners>
      <testrunner name="MbUnit"
                     typeName="MbUnit.AddIn.MbUnitTestRunner" 
                     assemblyPath="MbUnit.AddIn.dll"  />
      <testrunner name="NUnit"
                     typeName="NUnitAddin.NUnit.TestRunner.SimpleNUnitTestRunner" 
                     assemblyPath="NUnitAddin.NUnit.dll"  />
      ...
    </frameworktestrunners>
  </nunitaddin>
</configuration>

Every is now setup. NUnitAddIn will use the MbUnit.AddIn.MbUnitTestRunner class as test runner.

Getting started with ITestRunner

The last step of the job is to implement ITestRunner. We will name our runner accordingly to the name with have putted in the config file.

1. Create the class

   using NUnitAddIn.TestRunner.Framework;
   
   public class MbUnitTestRunner : ITestRunner, MarshalByRefObject
   {...}

2. Tag the class with the assemblies used by your test runner using the DependencyAttribute attribute. For MbUnit there are quite a few:

   [
   DependentAssembly("NUnitAddin.TestRunner"),
   DependentAssembly("NUnitAddin.TestRunner.Framework"),
   DependentAssembly("QuickGraph.Exceptions"), 
   DependentAssembly("QuickGraph.Concepts"),
   DependentAssembly("QuickGraph.Predicates"),
   DependentAssembly("QuickGraph.Collections"),
   DependentAssembly("QuickGraph.Representations"),
   DependentAssembly("QuickGraph.Algorithms"),
   DependentAssembly("QuickGraph.Serialization"),
   DependentAssembly("QuickGraph"),
   DependentAssembly("MbUnit.Core")
   ]
   public class MbUnitTestRunner : MarshalByRefObject, ITestRunner
   

   The two reference to NUnitAddin.TestRunner and NUnitAddIn.TestRunner.Framework are obligatory.
3. Make the object "long living" by making InitializeLifetimeService return null:

   public class MbUnitTestRunner : ITestRunner, MarshalByRefObject
   {
       public override Object InitializeLifeTimeService()
       {
           return null; 
       }
   }

4. ITestRunner define two methods, Abort and Run. Abort does not need to be implemented, so yoiu can throw a NotImplementedException in it. Run is where the job is done.

   public void Abort() 
   { 
       throw new NotImplementedException(); 
   } 
   
   public TestResultSummary Run( 
       ITestListener testListener, 
       ITraceListener traceListener, 
       string assemblyPath, 
       string testPath 
   ) 
   {
       ...
   }

   In the Run method, testListener is used to send test results to NUnitAddIn, ITraceListener is used to ouput messages to the console window, assemblyPath is the path to the test assembly and testPath is a string describing what has to be tested formatted as follows:

   ('N' | 'T' | 'M') : Type

   For example, values of testPath can be
   • N:MyTests.Tests, the namespace (and sub-namespaces) MyTests.Tests has to be run,
   • T:MyTests.Tests.SimpleFixture, the class SimpleFixture has to be run,
   • M:MyTests.Tests.SimpleFixture.SomeTest, the method SimpleFixture.SomeTest has to be run
   • null, the entire assembly is run
5. Let's start with an "hello world" test runner:

   public TestResultSummary Run( 
       ITestListener testListener, 
       ITraceListener traceListener, 
       string assemblyPath, 
       string testPath 
   ) 
   {
       traceListener.WriteLine("Hello World!");
   }
   

We are ready to make the first run of the Add-in. Open you dummy test project and right click either on the assembly, on a namespace, a type or a method and hit the "Run Tests..." rocket. If everything goes to plan, you should something like this appear in the output window:

------ Test started: Assembly: MbUnit.Tests.dll ------
Hello World!
---------------------- Done ----------------------

If you are lucky it worked out-of-the box, otherwize the next section deals with the Add-in debugging.

Debugging the Add-in

Different failure can appear at different levels. I have encountered several but hopefully, I had Jamie on my back helping me all the way. Here's a simple procedure:

1. Add a break point on the "Hello World" line inside the Run method,
2. Right-click on a test class, choose Test With... -> Debugger. If you are lucky, the debugger will hit the break point and you can do "classic" debugging.
3. If the debugger did not hit the break point, it is likely that NUnitAddIn failed to load the Add-In. You need to make the debugger break on exceptions:
   1. go to Debug -> Exceptions
   2. choose Commmon Language Runtime
   3. When exception is thrown, break into debugger
4. Run the tests again, this should give you the exception that make the Add-in loading fail.

Important note: when you recomile your project, you may have an error saying the assembly file cannot be copied because it is used by another process. At this point, you need to restart NUnitAddIn. To do this, right click on the rocket icon in the taskbar and click "Close". Recompile and yes it works!

Implementing the Run method

The rest of the work is mainly up to you. You need to load the assembly, look for the test fixture according to the "testPath" and execute them. The important thing is that the notification is done throught ITestListener.

Just for the fun, here's the output of the Add-in on the TestFixtureTest in MbUnit.Tests assembly:

------ Test started: Assembly: MbUnit.Tests.dll ------
C:\Documents and Settings\Peli\Mes documents\Tigris\mbunit\src\MbUnit.Tests\bin\StrongDebug\MbUnit.Tests.dll: [mbunit] Test Execution
[mbunit][setup] Load C:\Documents and Settings\Peli\Mes documents\Tigris\mbunit\src\MbUnit.Tests\bin\StrongDebug\MbUnit.Tests.dll assembly.
[mbunit][setup] Exploring types for fixtures.
[mbunit][setup] Setup Successfull, starting 1 tests.
[fixture] TestFixtureAttributeTest
[start] TestFixtureAttributeTest.SetUpMethod.TestMethod.TearDownMethod
[success] TestFixtureAttributeTest.SetUpMethod.TestMethod.TearDownMethod
[start] TestFixtureAttributeTest.SetUpMethod.FailedTest.TearDownMethod
[failure] TestFixtureAttributeTest.SetUpMethod.FailedTest.TearDownMethod
TestCase 'TestFixtureAttributeTest.SetUpMethod.FailedTest.TearDownMethod' failed: 
Equal assertion failed.
[[0]]!=[[1]]
MbUnit.Core.Exceptions.NotEqualAssertionException
C:\Documents and Settings\Peli\Mes documents\Tigris\mbunit\src\MbUnit.Core\Framework\Assert.cs(649,0): at MbUnit.Core.Framework.Assert.FailNotEquals(Object expected, Object actual, String format, Object[] args)
C:\Documents and Settings\Peli\Mes documents\Tigris\mbunit\src\MbUnit.Core\Framework\Assert.cs(208,0): at MbUnit.Core.Framework.Assert.AreEqual(Int32 expected, Int32 actual, String format, Object[] args)
C:\Documents and Settings\Peli\Mes documents\Tigris\mbunit\src\MbUnit.Core\Framework\Assert.cs(220,0): at MbUnit.Core.Framework.Assert.AreEqual(Int32 expected, Int32 actual)
c:\documents and settings\peli\mes documents\tigris\mbunit\src\mbunit.tests\testfixtureattributetest.cs(33,0): at MbUnit.Tests.TestFixtureAttributeTest.FailedTest()

1 succeeded, 1 failed, 0 skipped, took 0,00 seconds.


---------------------- Done ----------------------