Sunday, September 05, 2004

Reflector version: 4.1.4.0
Reflector.Graph version: 4.1.4.0
Download: www.dotnetwiki.org

Before getting the new version, make sure you have a look at the release notes below!

New Addins

This is the first release of the statement graph. It is surely not perfect. If you find anomalies in your graph, please prepare a code example that shows the problem and send it to me :)

posted on Sunday, September 05, 2004 12:13:00 PM UTC  #    Comments [4]
 Thursday, September 02, 2004

TestFixtureSetUp and TestFixtureTearDown

MbUnit now supports TestFixtureSetUp and TestFixtuteTearDown attribute to mark methods to be executed at the beginning of a fixture test case execution and at the end.

[TestFixture]
public class Fixture
{
    [TestFixtureSetUp]
    public void TestFixtureSetUp()
    {...}

    ...

    [TestFixturteTearDown]
    public void TestFixtureTearDown()
    {...}
}
  • Both method are optional and are allowed once per class,
  • If TestFixtureSetUp invocation fails, no test case are executed and they are all marked as failures,
  • TestFixtureTearDown is always invoked,
  • If TestFixtureTearDown fails, all the test cases are marked as failure
  • TestFixtureSetUp and TestFixtureTearDown are executed on the same instance. This is not ensured for the other methods,
  • Output in those methods are recorded in the reports

AssemblyCleanUp, SetUp and TearDown

MbUnit also support test assembly setup and teardown. Those methods should be enclosed in a class that is feeded to the AssemblyCleanUpAttribute (Assembly attribute, thanks Omer), They must be public and static. The setup method must be tagged with SetUpAttribute, and the teardown method with TearDownAttribute:

[assembly: AssemblyCleanUp(typeof(AssemblyCleaner))]
...
public class AssemblyCleaner
{
    [SetUp]
    public static void SetUp()
    {
        Console.WriteLine("Setting up {0}", typeof(AssemblyCleanUp).Assembly.FullName);
    }
    [TearDown]
    public static void TearDown()
    {
        Console.WriteLine("Cleaning up {0}", typeof(AssemblyCleanUp).Assembly.FullName);
    }
}
  • Only one class with AssemblyCleanUp per assembly is authorized,
  • Both methods are optional,
  • If SetUp fails, no tests are run and they are all marked as failure,
  • TearDown is always executed.
  • If TearDown fails, all the tests are marked as failure
posted on Friday, September 03, 2004 6:20:00 AM UTC  #    Comments [20]

This new Reflector.Addin creates a Method Graph where the vertex are methods, and the edges represent an invokation of a method. By clicking on the vertices, you jump to the next method, while keeping record of your previous steps.

Let's see this on FileStream.WriteByte method as shown above. The figure above is generated by right-clicking on the FileStream method and selecting "Method Invokation Graph". If I choose to to click FlushRight, the graph changes to:

 

Click on the next vertex will grow the tree, more and more...
posted on Thursday, September 02, 2004 8:17:00 AM UTC  #    Comments [1]
 Wednesday, September 01, 2004

I have revamped my TestFixture generator to work with the StatementGraph (i.e. FlowGraph). The generator logics has not changed:

  1. Build the Statement Graph from the method,
  2. Extract the path necessary to cover the entire graph (cover all statements in the method),
  3. Create one method for each path
  4. In each method documentation, output the decompiled code where the "target" statements have been highlighted.

Disclaimer

  1. This generator is far from behing finished or optimal: the algorithm to produce the graph is not optimal, I have not implement the Newyork Street Sweeper algorithm yet,
  2. If your code is buggy, the generator will generate tests for ... buggy code.

Examples

Here's a sample of what gets output from the examples in the StatementGraph article. Targetted statements are marked with /* i */ where i is the index of the tested statement

/// <summary>Test fixture for the &lt;see cref="StatementSamples"/&gt; class</summary>

/// <remarks />

[TestFixture()]

public class StatementSamplesTest

{

/// <summary>Tests the Simple method</summary>

/// <remarks>

/// <para>Test coverage (estimated): 100,0%</para>

/// <para>Target path:</para>

/// <code>/* 0 */ Console.WriteLine("hello");

/// </code>

/// </remarks>

[Test()]

[Ignore("NotImplemented")]

public virtual void Simple0()

{}

/// <summary>Tests the Block method</summary>

/// <remarks>

/// <para>Test coverage (estimated): 100,0%</para>

/// <para>Target path:</para>

/// <code>/* 0 */ Console.WriteLine("hello");

/// /* 1 */ Console.WriteLine("world");

/// </code>

/// </remarks>

[Test()]

[Ignore("NotImplemented")]

public virtual void Block0()

{}

/// <summary>Tests the If method</summary>

/// <remarks>

/// <para>Test coverage (estimated): 83,3%</para>

/// <para>Target path:</para>

/// <code>/* 0 */ if (value &lt; 0)

/// {

/// /* 1 */ Console.WriteLine("true");

/// /* 2 */ return;

/// }

/// Console.WriteLine("false");

/// </code>

/// </remarks>

[Test()]

[Ignore("NotImplemented")]

public virtual void If0()

{}

/// <summary>Tests the If method</summary>

/// <remarks>

/// <para>Test coverage (estimated): 50,0%</para>

/// <para>Target path:</para>

/// <code>/* 0 */ if (value &lt; 0)

/// {

/// Console.WriteLine("true");

/// return;

/// }

/// /* 1 */ Console.WriteLine("false");

/// </code>

/// </remarks>

[Test()]

[Ignore("NotImplemented")]

public virtual void If1()

{}

/// <summary>Tests the While method</summary>

/// <remarks>

/// <para>Test coverage (estimated): 100,0%</para>

/// <para>Target path:</para>

/// <code>/* 0 */ int num1 = 0;

/// while ((num1 &lt; 10))

/// {

/// /* 1 */ Console.Write(num1++);

/// }

/// </code>

/// </remarks>

[Test()]

[Ignore("NotImplemented")]

public virtual void While0()

{}

}

 

posted on Thursday, September 02, 2004 2:20:00 AM UTC  #    Comments [3]
 Tuesday, August 31, 2004

This post presents a new Reflector Addin that creates and diplays the graph of statement inside methods: the StatementGraph. This addin is an evolution of the IL graph. (this addin is not yet available for download).

 The statement graph is built from the Reflector CodeModel where each IStatement instance is a vertex and edges are added accordingly to the code "flow" (creating the edges is the most involved task). The rest of the job is handled by the QuickGraph library. When it makes sense, the vertices are clickable and you can jump to the invoked method, etc... The next step will be to update the Automatic Unit Test generator with the StatementGraph...

Let's see some simple methods and their corresponding graphs:

Simple

  • Original code:
    public void Simple()
    {
        Console.WriteLine("hello");
    }
    
  • Decompiled:
    public void Simple()
    {
          Console.WriteLine("hello");
    }
  • Graph:

Two statements in sequence

  • Original code:
    public void Body()
    {
        Console.WriteLine("hello");
        Console.WriteLine("world");
    }
    
  • Decompiled:
    public void Body()
    {
          Console.WriteLine("hello");
          Console.WriteLine("world");
    }
  • Graph:

If - Then - Else

  • Original code:
    public void If(int value)
    {
        if (value<0)
            Console.WriteLine("true");
        else
            Console.WriteLine("false");
    }
  • Decompiled:
    public void If(int value)
    {
          if (value < 0)
          {
                Console.WriteLine("true");
                return;
          }
          Console.WriteLine("false");
    }
    
  • Graph:

While

  • Original code:
    public void While()
    {
        int i = 0;
        while(i<10)
        {
            Console.Write(i++);
        }
    }
  • Decompiled:
    public void While()
    {
          int num1 = 0;
          while ((num1 < 10))
          {
                Console.Write(num1++);
          }
    }
  • Graph:

While with break and continue

  • Original code:
    public void WhileBreakContinue()
    {
        int i = 0;
        while (i < 10)
        {
            if (i == 5)
                continue;
            if (i == 7)
                break;
            Console.Write(i++);
        }
        Console.WriteLine("Finished");
    }
  • Decompiled:
    public void WhileBreakContinue()
    {
          int num1 = 0;
          while ((num1 < 10))
          {
                if (num1 == 5)
                {
                      continue;
                }
                if (num1 == 7)
                {
                      break;
                }
                Console.Write(num1++);
          }
          Console.WriteLine("Finished");
    }
    
    
    
  • Graph:

Try - Catch

  • Original code:
    public void TryCatch()
    {
        try
        {
            Console.WriteLine("hello");
        }
        catch (Exception ex)
        {
            Console.WriteLine("Boom: {0}",ex);
        }
    }
    
  • Decompiled:
    public void TryCatch()
    {
          try
          {
                Console.WriteLine("hello");
          }
          catch (Exception exception1)
          {
                Console.WriteLine("Boom: {0}", exception1);
          }
    }
  • Graph:

posted on Wednesday, September 01, 2004 6:40:00 AM UTC  #    Comments [18]
 Monday, August 30, 2004

Just finished the private presentation of the PhD... One month to go and I'll get the degree :)

For the curious, here's a link to the presentation: http://blog.dotnetwiki.org/downloads/PhDPresentationdeHalleux.pdf

posted on Tuesday, August 31, 2004 4:00:00 AM UTC  #    Comments [6]
 Sunday, August 29, 2004

The "classic" TestFixture now supports a new type of test case: combinatorial tests. This fixture comes from an original idea of Jamie Cansdale, and it's design has been refined with the joint help of Jamie, Omer van Kloeten and the dev@mbunit.tigris.org mailing list.

A simple example

Suppose that we have a ICountX interface that defines a method that counts the number of 'x' in a string:

interface ICountX
{
   int Count(string s);
}

You have two classes that implement that interface:

class SickCountX : ICountX
{
   public int Count(string s)
   {
       return 2;
   }
}

class CountX : ICountX
{
   public int Count(string s)
   {
       int count = 0;
       foreach(char c in s)
           if (c=='x')
              count++;
       return count;
   }
}

Now you would like to test that those two interface implementations work correctly. Let's start by writing a test method inside a TestFixture. The CountIsExact method receives a string, the number of x in the string and checks that a provided ICountX implementation computes the expected x count:

[TestFixture]
public class ICountXTest
{
   public class StringX
   {
       public StringX(string value, int xcount){Value = value; XCount = xcount;}
       public string Value;
       public int XCount;
       public override string ToString() { return String.Format("{0},{1}",this.Value,this.XCount);}
   }

   [CombinatorialTest]
   public void CountIsExact(
      ICountX counter,
      StringX s
      )
   {
       Assert.AreEqual(s.XCount, counter.Count(s.Value));
   }
}

Next, we create a few strings and xcount that can serve for creating test case. By simplicity we use the new C# 2.0 iterators to implement that:

[Factory(typeof(StringX))]
public IEnumerable Strings()
{
    yield return new StringX("",0);
    yield return new StringX("x",1);
    yield return new StringX("xa",1);
    yield return new StringX("xax",2);
    yield return new StringX("aaa",0);
}

Note that we have tagged the method with Factory so that MbUnit knows this method creates StringX instances. We also create another method that will create instance of ICountX to test:

[Factory(typeof(ICountX))]
public IEnumerable Counters()
{
   yield return new SickCountX();
   yield return new CountX();
}

Now, we would like to cross product the output of Counters and StringX and feed the combination in the CountIsExact test method. To do so, we use the UsingFactories attribute to tag the parameters of the method:

[CombinatorialTest]
public void CountIsExact(
   [UsingFactories("Counters")] ICountX counter,
   [UsingFactories("Strings")] StringX s
)
{...}

The fixture is ready, so we can give it a run. The output of the test execution is as follows:

[mbunit][Failure] CountIsExact(Counters(SandBox.SickCountX),Strings(,0))
[mbunit][Failure] CountIsExact(Counters(SandBox.SickCountX),Strings(x,1))
[mbunit][Failure] CountIsExact(Counters(SandBox.SickCountX),Strings(xa,1))
[mbunit][Success] CountIsExact(Counters(SandBox.SickCountX),Strings(xax,2))
[mbunit][Failure] CountIsExact(Counters(SandBox.SickCountX),Strings(aaa,0))
[mbunit][Success] CountIsExact(Counters(SandBox.CountX),Strings(,0))
[mbunit][Success] CountIsExact(Counters(SandBox.CountX),Strings(x,1))
[mbunit][Success] CountIsExact(Counters(SandBox.CountX),Strings(xa,1))
[mbunit][Success] CountIsExact(Counters(SandBox.CountX),Strings(xax,2))
[mbunit][Success] CountIsExact(Counters(SandBox.CountX),Strings(aaa,0))

6 succeeded, 4 failed, 0 skipped, took 0,00 seconds.

It worked! As expected, we have 5 x 2 = 10 test case generated. Each test case name is generated out of the data source and different data values. Of course, you can "bind" data to the parameters using various other ways and the tests support more that 2 parameters as we will see in the following. We could also have added multiple Using Attributes.

Prequisites

This new features uses the TestFu.Operations framework in the background. This framework is detailed in the here (part 1) and here (part 2).

Rationale

The CombinatorialTest has the following workflow:

  • for each parameter of the test method, get all the domains D of the parameter from the UsingAttributes (there are a bunch of those). This creates a collection of domain for each parameter which is also a domain itself, which we'll call parameter domain: PD = collection of D,
  • foreach tuple in the CartesianProduct of the parameter domains
    • foreach ptuple in the PairWizeProduct of the domain in the tuple (each entry is a domain for the corresponding method parameters)
      • Create a test case that will invoke the test method with the values in ptuple.

Note that in the example above, PD = { { Counters }, { Strings } }, hence the Cartesian product has only one element {Counters} x {Strings} = (Counters,Strings). 

Custom Using Attributes

All Using attribute inherit from the abstract base class attribute UsingBaseAttribute which defines one abstract method:

public abstract class UsingBaseAttribute : Attribute
{
    public abstract void GetDomains(
        IDomainCollection domains, 
        ParameterInfo parameter,
        object fixture);
}

This method receives the tagged parameter, an instance of the fixture class and a collection of domains. It can add new domain to this fixture. For example, the following attribute, UsingLinear, generate a linearly spaced vector of integers:

public sealed class UsingLinearAttribute : UsingBaseAttribute
{
    private IDomain domain;
    public UsingLinearAttribute(int start, int stepCount)
    {
        this.domain = new LinearInt32Domain(start, stepCount);
    }
    public override void GetDomains(IDomainCollection domains, ParameterInfo parameter, object fixture)
    {
        domains.Add(domain);
    }
}

Here we have used the LinearInt32Domain shipped with TestFu.Operations and we have added that domain to the domain collection. Let's write a test that uses that attribute and see the result:

[CombinatorialTest]
public void UsingLinear(
    [UsingLinear(0, 10)] int i)
{
    Console.WriteLine(i);
}


--- output
[mbunit][Success] UsingLinear(0)
[mbunit][Success] UsingLinear(1)
[mbunit][Success] UsingLinear(2)
[mbunit][Success] UsingLinear(3)
[mbunit][Success] UsingLinear(4)
[mbunit][Success] UsingLinear(5)
[mbunit][Success] UsingLinear(6)
[mbunit][Success] UsingLinear(7)
[mbunit][Success] UsingLinear(8)
[mbunit][Success] UsingLinear(9)
10 succeeded, 0 failed, 0 skipped, took 0,00 seconds.

Built in Using Attributes

MbUnit comes with a number of built-in using attribute that should get you started almost all situations.

UsingLinearAttribute

Already described previously.

UsingLiteralsAttribute

This attribute lets you specify a list of value separated by ';'. MbUnit will try to convert those to the parameter type using Convert.ChangeType method:

[CombinatorialTest]
public void UsingLinearAndLiterals(
    [UsingLinear(0, 3)] int i,
    [UsingLiterals("a;b")] char c
)
{
    Console.WriteLine("{0} {1}",i,c);
}
--- output
[mbunit][Success] UsingLinearAndLiterals(0,a)
[mbunit][Success] UsingLinearAndLiterals(0,b)
[mbunit][Success] UsingLinearAndLiterals(1,a)
[mbunit][Success] UsingLinearAndLiterals(1,b)
[mbunit][Success] UsingLinearAndLiterals(2,a)
[mbunit][Success] UsingLinearAndLiterals(2,b)
6 succeeded, 0 failed, 0 skipped, took 0,00 seconds.

UsingFactoriesAttribute

This attribute will look for factory method in a specified type. If the factory type is not specified, it will default it to the fixture type. If the member name is not specified, it will look for all the methods tagged with Factory attribute and compatible with the parameter type:

  • Not specifying the member name, MbUnit looks for all the compatible factories:
    [Factory]
    public int GetZero()
    {
        return 0;
    }
    [Factory]
    public int[] GetZeroOne()
    {
        return new int[] { 0, 1 };
    }
    [Factory(typeof(int))] // we need to specify the factory type
    public IEnumerable GetZeroOneAsEnumerable()
    {
        return this.GetZeroOne();
    }
    [CombinatorialTest]
    public void UsingFactories(
        [UsingFactories] int i
        )
    {
        Console.WriteLine("{0}", i);
    }
    
    
    -- output
    
    [mbunit][Success] UsingFactories(GetZero(0))
    [mbunit][Success] UsingFactories(GetZeroOne(0))
    [mbunit][Success] UsingFactories(GetZeroOne(1))
    [mbunit][Success] UsingFactories(GetZeroOneAsEnumerable(0))
    [mbunit][Success] UsingFactories(GetZeroOneAsEnumerable(1))
    
    
  • Using external factories by specifying a factory type:
    public class IntFactory
    {
        [Factory]
        public int One()
        {
            return 1;
        }
    }
    
    
    [CombinatorialTest]
    public void UsingFactory(
        [UsingFactories(typeof(IntFactory))] int i)
    {
        Console.WriteLine("{0}", i);
    }
    
    --- output
    
    [mbunit][Success] UsingFactory(One(1))
    1 succeeded, 0 failed, 0 skipped, took 0,00 seconds.
    

UsingImplementationsAttribute

This attribute looks for all the implementation of a type in an assembly, creates an instance and feeds it to the test.

[CombinatorialTest]
public void TestCountX([UsingImplementationsAttribute(typeof(ICountX))] ICountX countX)

What about tuple filtering ?

In some situation you may want to filter out tuple. For example, we want to check that a methods throws ArgumentNullException on null arguments:

[Factory]
public string[] Strings()
{
    return new string[] { null, "hello" };
}

[CombinatorialTest]
[ExpectedArgumentNullException]
public void TestWithValidator(
    [UsingFactories("Strings")] string s1,
    [UsingFactories("Strings")] string s2
)
{
    if (s1 == null)
        throw new ArgumentNullException();
    if (s2 == null)
        throw new ArgumentNullException();
}

(Note that ExceptedException and all the other decorator work on the combinatorial tests). If we run the test now, we wil have unwanted tuple such as (null, null) and ("hello", "hello"): the first does not give much information and the second will not throw and hence, is erroneous. In this kind of situation, you can specify a "filter" method that takes the same arguments as the test method and return bool:

public bool IsValid(string s1, string s2)
{
   if (s1 == null && s2 == null)
      return false;
   if (s1 != null && s2 != null)
      return false;
   return true;
}

[CombinatorialTest(TupleValidatorMethod = "IsValid")]
[ExpectedArgumentNullException]
public void TestWithValidator(...

--- output

[mbunit][Success] TestWithValidator(Strings(),Strings(hello))
[mbunit][Success] TestWithValidator(Strings(hello),Strings())

As expected, the unwanted tuple have been dropped.

Conclusion

Combinatorial test provides a flexible, extensible and powerfull way of generating test case. It implements the classic PairWize combinatorial generation in order to avoid exponential test explosion while integrating smoothly in the MbUnit architecture.

posted on Monday, August 30, 2004 5:51:00 AM UTC  #    Comments [4]
 Saturday, August 28, 2004

Reflector version: 4.1.2.0
Reflector.Graph version: 4.1.2.0
Download: www.dotnetwiki.org

Before getting the new version, make sure you have a look at the release notes below!

Important Changes:

  • The graphs are now visualized in SVG by embedding Internet Explorer inside Reflector. Make sure you have a SVG viewer that works with IE installed on your machine. I strongly recommend Adobe SVG Viewer.
  • The version number of the assemblies is now following Reflector version. Hence, current version number is 4.1.2.0.

Release Notes:

  • The Assembly Graph and IL graph now have clickable nodes. In general, clicking on a vertex will move the selection to that element in the Reflector tree.
  • The Assembly Graph now shows all the referenced even if they are not loaded. Click on a unloaded assembly will load it in Reflector,

Adobe SVG viewer tips:

If you are using Adobe SVG viewer, here are a few tips:

  • Alt + left down + move move = panning the view,
  • Ctrl + left click = zoom in,
  • Ctrl + Shift + left click = zoom out,
  • Ctrl + left click + mouse move = zoom region,
  • Right click pops up the SVG viewer context menu.

Here's the new look of the Assembly graph.

posted on Sunday, August 29, 2004 2:38:00 AM UTC  #    Comments [15]

In the previous post on combinatorial testing, I mentionned that Cartesian product was not a good solution when the number of domains and their dimension grew, because of obvious explosion of the number of tests.

A common approach to this problem is to consider a set of combination to create all the possible 2-tuples between the domains. It is called  PairWize or AllPairs suites generation. In TestFu.Operations, I decided to implement the algorithms found in [1]. Currently only the A1 algorithm is implemented. A2 and A4 should follow soon. Note that those algorithms are constructive, memory cheap and fast.

PairWize suite generation

The use of the PairWize algorithm is straightforward and follows the same semantics as the cartesian product. In the example below, we want to generate the allpairs for 3 domains of respective size 2,3,4.

int[] array1 = new int[] { 1, 2 };
char[] array2 = new char[] { 'a', 'b','c' };
string[] array3 = new string[] { "a", "combinatorial", "hello","world" };

int i = 1;
foreach (ITuple tuple in Products.PairWize(array1, array2, array3))
{
    Console.WriteLine("{0}: {1}",i++, tuple);
}

-- output
1: 1, a, a
2: 1, a, combinatorial
3: 2, a, a
4: 1, a, hello
5: 1, a, a
6: 1, a, world
7: 2, a, a
8: 2, b, a
9: 1, b, combinatorial
10: 2, b, combinatorial
11: 2, b, hello
12: 1, b, combinatorial
13: 2, b, world
14: 2, b, combinatorial
15: 1, c, a
16: 1, c, hello
17: 1, c, combinatorial
18: 2, c, hello
19: 1, c, hello
20: 1, c, world
21: 2, c, hello
22: 2, c, a
23: 1, c, world
24: 2, c, combinatorial
25: 2, c, world
26: 2, c, hello
27: 1, c, world
28: 2, c, world

If you wish to compare with the article, this is the output for a (k,l) = (7,3) and for the bipartite graphs A1 = {0,1,2,3} B1={4,5,6}, A2 = {0,4,2,3} B2={1,5,6} and A3 = {0,1,5,3} B3={4,2,6}. Remember that if we want to generate the test cases using the Cartesian product, we would have 3^7 = 2187 tests to run.

1: 0, 0, 0, 0, 0, 0, 0
2: 0, 0, 0, 0, 1, 1, 1
3: 1, 0, 0, 0, 0, 1, 1
4: 0, 1, 0, 0, 1, 0, 1
5: 0, 0, 0, 0, 2, 2, 2
6: 2, 0, 0, 0, 0, 2, 2
7: 0, 2, 0, 0, 2, 0, 2
8: 1, 1, 1, 1, 0, 0, 0
9: 0, 1, 1, 1, 1, 0, 0
10: 1, 0, 1, 1, 0, 1, 0
11: 1, 1, 1, 1, 1, 1, 1
12: 1, 1, 1, 1, 2, 2, 2
13: 2, 1, 1, 1, 1, 2, 2
14: 1, 2, 1, 1, 2, 1, 2
15: 2, 2, 2, 2, 0, 0, 0
16: 0, 2, 2, 2, 2, 0, 0
17: 2, 0, 2, 2, 0, 2, 0
18: 2, 2, 2, 2, 1, 1, 1
19: 1, 2, 2, 2, 2, 1, 1
20: 2, 1, 2, 2, 1, 2, 1
21: 2, 2, 2, 2, 2, 2, 2

Compairing results with AllPairs

For 10 domains with 10 values each, the TestFu algorithm generate 370 test suite. The AllPairs tool generate 177 which is roughly 50% less. This behavior was expected by the algorithm author. A greedy algorithm on the output could lower our result. Note that the tests were generate in 36.419ms which shows the efficiency of the algorithm.

References:

[1] Efficient Algorithms for Generation of Combinatorial Covering Suites, by Adrian Dumitrescu

posted on Saturday, August 28, 2004 7:09:00 AM UTC  #    Comments [6]
 Friday, August 27, 2004

TestFu has now limited support for generate test suites using Combinatorial Testing as the TestFu.Operations namespace. Combinatorial Testing is an interresting testing technique that has been studied a quite some time now (see citeseer search). There already exists a few package that provide tools to generate test suites (see jenny, allpairs) but none in the .Net framework.

Combinatorial Testing description and Glossary

In this chapter, I will define the different "actors" that interact in the combinatorial testing. I will link them to their corresponding interface in TestFu.

  • A domain (IDomain) is a finite set of objects.
    • Ex: {1,2,3} is a domain composed of the integers 1,2,3
  • The boundary of a domain D is the set of objects in D that are at the boundary of the domain.  The boundary is also a domain,
    • If we decide that the first and last element of an array is it's domain, {1,3} is the boundary of {1;2,3}
  • A tuple (ITuple) acts as a container for other objects,
  • A product on a domain collection (IDomainCollection) is an algorithm (ITupleEnumerable) that generate a suite of tuples (ITupleEnumerator), created by combining one element of each domain, using a predefined strategy.

Currently, the only product available is the cartesian product which returns an exhaustive enumeration of the all possible combination. It is well-known that this product is not efficient at all covering the tests because of the exponential explosion of the number of suites, better strategy have been developped such as the PairWaise covering (all pairs), t-wize, etc... I haven't had the time to look at those yet.

Let's see some code

The TestFu.Operations namespace is designed to be simpler but no simpler. Let's start by defining some domains:

int[] array1 = { 1, 2 };
string[] array2 = { "a", "combinatorial", "hello", "world" };
char[] array3 = { 'a', 'b', 'c' };

Note: TestFu.Operations currently supports arrays, collection (ICollection), enumerable collection (IEnumerable) or singleton domain (1 element). Of course, this is totally extendible. You could easily think about a domain reading the rows of a DataTable, or the elements of a XmlDocument

The static class Products contains a rich set of methods that can create the different products. Let's start with the cartesian product of array1 x array2:

int i = 1;
foreach (ITuple tuple in Products.Cartesian(array1, array2))
{
    Console.WriteLine("\t{0}: {1}",i++,tuple);
}

-- output
        1: 1, a
        2: 1, combinatorial
        3: 1, hello
        4: 1, world
        5: 2, a
        6: 2, combinatorial
        7: 2, hello
        8: 2, world

As expected, each returned tuple is a collection of element from each domain. We could also decide to test the boundaries of the arrays only (first and last element each time):

i = 1;
foreach (ITuple tuple in Products.BoundaryCartesian(array1, array2))
{
    Console.WriteLine("\t{0}: {1}", i++, tuple);
}

-- output
        1: 1, a
        2: 1, world
        3: 2, a
        4: 2, world

The products can act on an arbitrary number of domains so we can add array3 in the product (don't forget about the explosion of the tests number!):

i = 1;
foreach (ITuple tuple in Products.Cartesian(array1, array2, array3))
{
    Console.WriteLine("\t{0}: {1}", i++, tuple);
}

-- output
        1: 1, a, a
        2: 1, a, b
        3: 1, a, c
        4: 1, combinatorial, a
        5: 1, combinatorial, b
        6: 1, combinatorial, c
        7: 1, hello, a
        8: 1, hello, b
        9: 1, hello, c
        10: 1, world, a
        11: 1, world, b
        12: 1, world, c
        13: 2, a, a
        14: 2, a, b
        15: 2, a, c
        16: 2, combinatorial, a
        17: 2, combinatorial, b
        18: 2, combinatorial, c
        19: 2, hello, a
        20: 2, hello, b
        21: 2, hello, c
        22: 2, world, a
        23: 2, world, b
        24: 2, world, c

Where to go from here ?

The next important step is to implement an algorithm that computes the pair-wize or t-wize suite generation. This is very important if you have a lot of domains. In a near future, I will also show how we can use these products to produce a new type of fixture in MbUnit. This fixture was suggested by Jamie Cansdale.

In this post, I have showed a new, simple, way of creating combinatorial tests using the TestFu.Operations namespace.

posted on Saturday, August 28, 2004 3:07:00 AM UTC  #    Comments [5]
 Thursday, August 26, 2004

Yesterday, Jamie Cansdale (NunitAddin) and I spent time "pair-programming" on MbUnit, NUnitAddIn and other tools like the Reflector.Graph addins. The collaboration has been very productive! I've got a lot of blog entry to write about the things we have built yesterday, I'll summarize quickly:

  • MbUnit has now a much better support for NUnitAddIn:
    • this solution should be "version" proof: it should not break if NUnitAddIn updates and so on,
    • you can select a method and execute the tests involving that method,
    • MbUnit generates the Html report and puts the URL in the output, ready for you
  • MbUnit has new fixtures! CrossFixture generates generate the cross products of different data source and hands it to the test...
  • Reflector.Graph renders to SVG: we are now hosting IE inside Reflector to display SVD graphs...
  • Reflector.Graph graphs has clickable links! You can click on an assembly to load it, and so on...

A lot of work, a lot of fun...

 

posted on Friday, August 27, 2004 2:27:00 AM UTC  #    Comments [0]
 Tuesday, August 24, 2004

In this entry, I'll give a tutorial on using QuickGraph to compute the maximum flow of a capacitated network. I will start by giving some defnitions in order to make things clear, then I will apply the QuickGraph algorithms to a practical example.

Maximum Flows

Computing the maximum flow in a network is one of the classic graph theory problem (see [1]). Here's a prototype of flow network:

Imagine a pipeline network for transporting oil from a single source to a single sink. Each edge represents a section of pipeline, and the endpoints of the edge corresponds to the junctures at the ends of that section. The edge capacity is the maximum amount of oil that can flow through the corresponding section per unit time.

What is the maximum amount of oil per unit time that can flow through the network ? That question is a maximum flow problem.

A single-source single-sink network is a connected directed graph that has a distinguised vertex called the source with nonzero outdegree, and a distinguished vertex called sink with  nonzero indegree. A single source-single sink network with source s and sink t (also called target) is called a s-t network. A capacitated network is a connected directed graph such that each edge e is assigned a nonnegative weight cap(e), called the capacity of edge e.

In the following, we shall consider a capacitated s-t network.

Sample network

I will illustrate the example on a 5-vertex capacitated network with source s and sink t as depicted below. This network is extracted from Chapter 12 of [1].

By convention, each edge is tagged with it's capacity (on the left) and, if computed, the flow. s is the source, t is the sink.

Building the sample network with QuickGraph

The first step of the tutorial is to create the sample network above using QuickGraph. To do so, we create a AdjacencyGraph and populate it "manually" with the vertices and edges:

using QuickGraph; // NamedVertex
using QuickGraph.Providers; // providers
using QuickGraph.Representations; // AdjacencyGraphg
using QuickGraph.Concepts; // IEdge

// the graph
AdjacencyGraph graph = new AdjacencyGraph(
    new NamedVertexProvider(),
    new EdgeProvider(),
    true);
// the capacity dictionary
EdgeDoubleDictionary capacities = new EdgeDoubleDictionary();

// adding vertices
NamedVertex s = (NamedVertex)graph.AddVertex(); s.Name = "s";
NamedVertex x = (NamedVertex)graph.AddVertex(); x.Name = "x";
NamedVertex v = (NamedVertex)graph.AddVertex(); v.Name = "v";
NamedVertex w = (NamedVertex)graph.AddVertex(); w.Name = "w";
NamedVertex t = (NamedVertex)graph.AddVertex(); t.Name = "t";

// adding edges
IEdge sx = graph.AddEdge(s, x); capacities[sx] = 5;
IEdge sv = graph.AddEdge(s, v); capacities[sv] = 7;
IEdge xv = graph.AddEdge(x, v); capacities[xv] = 3;
IEdge xw = graph.AddEdge(x, w); capacities[xw] = 7;
IEdge wv = graph.AddEdge(w, v); capacities[wv] = 5;
IEdge wt = graph.AddEdge(w, t); capacities[wt] = 4;
IEdge vt = graph.AddEdge(v, t); capacities[vt] = 6;

Once the graph is built, you can easily draw it using GraphvizAlgorithm. This topic has already been discribed in previous posts so I won't enter into the details .

Preparing the network for the Maximum Flow

In order to work properly, the maximum flow algorithms require that for each edge (u,v) in the network, there exists a reversed edge (v,u). Moreover, a dictionary associating each edge to its reversed must be provided. The sample network obviously does not fullfill this requirement. Therefore, we must add "articifial" reversed edge that have capacity equal to zero. In order to help you with this (tedious) task, QuickGraph provides the ReversedEdgeAugmentorAlgorithm algorithm that will do the job for you:

using QuickGraph.Algorithms.MaximumFlow;

// creating the augmentor
ReversedEdgeAugmentorAlgorithm reversedEdgeAugmentor = new ReversedEdgeAugmentorAlgorithm(this.graph);
// attaching handler to the ReversedEdgeAdded event
// this event is triggered on each "artifical" edge added to the graph
ReversedEdgeAugmentorAlgorithm reversedEdgeAugmentor.ReversedEdgeAdded += new EdgeEventHandler(reversedEdgeAugmentor_ReversedEdgeAdded);

// add the articifial edges
reversedEdgeAugmentor.AddReversedEdges();
...
// cleans up
reversedEdgeAugmentor.RemoveReversedEdges();

// this handler sets the capacity of the new edge to zero
void reversedEdgeAugmentor_ReversedEdgeAdded(Object sender, EdgeEventArgs e)
{
    capacities[e.Edge] = 0;
}

The method AddReversedEdges augments the graph with the reversed edges, while the RemoveReversedEdges "cleans up the mess". If we draw the flow graph after the AddReversedEdges call, it gives the following results (where the reversed edges are dashed):

Computing the MaximumFlow

QuickGraph implements 2 maximum flow algorithms: Edmund Karp and Push Relabel. We will use PushRelabel as it is more efficient, however, both inherit from the abstract base class MaximumFlowAlgorithm. The method Compute computes the maximum flow and returns it's value:

MaximumFlowAlgorithm algo = new PushRelabelMaximumFlowAlgorithm(
    graph,
    capacities,
    reversedEdgeAugmentor.ReversedEdges
    );

double value = algo.Compute(s, t);

After cleaning up the reversed edges, we can draw the graph with the maximum flow in each edge. Note that for this sample, the maxium flow value is 10.

Demo source

The full source of the demo is available in the MbUnit CVS at http://mbunit.tigris.org/source/browse/mbunit/src/QuickGraphTest/MaximumFlowDemo.cs

Acknoledgment

Many thanks to Arun Bhalla for porting PushRelabel to C#

References

[1] Graph Theory and Its applications, Jonathan Gross and Jay Yellen

posted on Tuesday, August 24, 2004 11:02:00 PM UTC  #    Comments [7]
 Monday, August 23, 2004

As mentionned in a previous post, I mentionned that dnAnalytics was a new project providing a managed wrapper above LAPACK. Those guys have been doing a great job and the current release (trunk 271) is already featuring a lot of the "classic" Matrix Algebra algorithms:

  • Dense matrices (float or double) with support for the classic operations, +, *, -,
  • Norms: norm-1, norm-2, norm-infty, etc...,
  • Condition number,
  • QR decomposition,
  • LU decomposition,
  • SVD decomposition

The library is both accessible as a fully managed version, and a managed wrapper above the native, rock solid, LAPACK library. If you are doing matrix algebra in your application, and you care about robustness, efficiency and accuracy, you should be interrested by this project.

 

posted on Tuesday, August 24, 2004 1:22:00 AM UTC  #    Comments [0]
 Sunday, August 22, 2004

In my previous Singleton Unit Testing, I proposed a possible solution for unit testing a singleton (I'll name it the ShortLivingSingleton approach). Omer Van Kloeten came up with another approach (the KillTheSingleton approach), Darren Oakey with the CreateForTesting approach and , Len Holgate finishes the picture tearing appart our the first 2 solutions (the SingletonIsBad approach). Since comments on this topic have been interresting, they are worth a little summary.

The problem

The application you are developping is using a Singleton, that you need to test. How do you test a singleton ?

SingletonIsBad approach

  1. Singleton are evil! If you have the opportunity to refactory the code, make sure you definitely need this singleton, otherwize get rid of it.
  2. You have no choice and you cannot get rid of the singleton. Make sure you split the singleton functionalities in two classes: The class that does all the work and the singleton aspect that prevents multiple instances. Doing so, you will not have to cheat to test the singleton.
  3. You could not fullfill any of the point above, and you will need to cheat...

The two first point came from Len Holgate and I totally agree with him. The third point is for tester who do not have the possiblity of avoiding or refactoring the singleton. To tackle this problem, I and Omer have proposed 2 cheats and Darrel Oakley proposed a wider solution to the problem.

CreateForTesting approach

The approach asks you to embed some functionality in a class that helps other people test it. The most common thing is to add - to most objects - a function CreateTestFoo, where Foo is the object that you are testing. This gives you a new and fully initialized version of Foo - however complicated Fooactually is.

Cheating the singleton.

Both cheat solutions are based on Reflection, therefore they will not work if you have denied Reflection on your tested assembly (Haacked's  comment).

ShortLivingSingleton  approach

This approach consists of creating a new non-singleton approach for each test and do the testing on this instance. To do so, we use reflection to access the private constructor and instanciate the singleton. Therefore, the singleton is never created and never used.

KillTheSingleton approach

This approach kills the singleton after each test, in the TearDown method for example. To "kill" the singleton, the static field holding the singleton reference is nullified using Reflection and garbage collection is forced.

Conclusions

Singletons are evil but yet you find people using them (shame on me). They are problematic to test because you cannot separate the unit tests.  The first step for testing them is actually to refactor your application and get rid of the singleton. If this cannot be done, you have to cheat the singleton using Reflection to properly test them.

ps: I'd like to thank all the people who commented the blog entry for their constructive comments :) Cheers, Jonathan

posted on Monday, August 23, 2004 1:06:00 AM UTC  #    Comments [6]
 Thursday, August 19, 2004

Here are the NSort benchmarks for arrays of integer filled with random data. These benchmarks are for the non-generic versions.

The code to generate the benchmarks uses NPerf is given below:

using System;
using System.Collections;
using NPerf.Framework;

namespace NSort.Perf
{
    [PerfTester(typeof(ISorter), 10
        , Description = "Sort Algorithm benchmark for Random data"
        , FeatureDescription = "Collection size")]
    public class SorterBenchmark
    {
        protected int[] list;
        public int CollectionCount(int testIndex)
        {
            int n = 0;
            if (testIndex < 0)
                n=10;
            else
                n = (testIndex+1)*500;
            return n; 
        }
        [PerfRunDescriptor]
        public double RunDescription(int testIndex)
        {
            return (double)CollectionCount(testIndex);
        }

        [PerfSetUp]
        public override void SetUp(int testIndex, ISorter sorter)
        {
            Random rnd = new Random();
            this.list = new int[CollectionCount(testIndex)];
            for (int i = 0; i < this.list.Length; ++i)
                this.list[i] = rnd.Next();
        }
        [PerfTest]
        public void SortRandomData(ISorter sorter)
        {
            sorter.Sort(this.list);
        }
    }
}

 

posted on Thursday, August 19, 2004 11:48:00 PM UTC  #    Comments [0]
 Wednesday, August 18, 2004

NSort is a very nice and flexible package of sorting (15) algorithms from which the developer can choose. This library was a jointly work from me, Marc Clifton and Robert Rohde. In this blog, I'll show how NSort can be quickly "upgraded" to Generic and how you can use the extensibility of MbUnit to test it efficiently.

Generic

Converting the algorithms to Generic was really straightforward. For almost all cases, it was just a matter of adding <T> here and there, and replacing temporary object instance by T. Let's see this with the two interfaces of the project: IStorter and ISwap:

using System.Collections;
...

public interface ISorter
{
    IComparer Comparer {get;}
    void Sort(IList list);
}
public interface ISwap
{
    void Swap(IList array, int left, int right);
    void Set(IList array, int left, int right);
    void Set(IList array, int left, object obj);
}

Now, their generic brothers:

using System.Collections.Generic;
...

public interface ISorter<T>
{
    IComparer<T> Comparer {get;}
    void Sort(IList<T> list);
}
public interface ISwap<T>
{
    void Swap(IList<T> array, int left, int right);
    void Set(IList<T> array, int left, int right);
    void Set(IList<T> array, int left, object obj);
}

That was pretty quick. The conversion of the algorithms followed the same ideas and a dozens of cut/paste/replace later the NSort.Generic namespace was born.

Testing

There is a well-know proverb that says "the one who live by the cut-and-paste, die by the cut-and-paste" and that's exactly how I ported NSort to generics... so to ensure the quality of it really needs proper testing.

Testing NSort

Let's start with the testing of the non-generic classes. The main purpose of the NSort is to provide implementation of the ISorter interface that effectively sort list of elements and that's what we are willing to test. A quick (internal) brain strom for test cases of ISorter yields:

  • Sort a null list throws ArgumentNullException,
  • Sort a list with 0,1,2, n (n large) elements check it is effectively sorted. The purpse of the 0,1,2 is to test "boundary" lists.

Since we are testing an interface, this is a good candidate to use CompositeUnitTesting the TypeFixture of MbUnit:

[TypeFixture(typeof(ISorter))]
public class SorterTest
{
    private int[] list;
    private void CreateSortAndCheckSorted(ISorter sorter, int length)
    {
        Random rnd = new Random();
        list = new int[length];
        // create data
        for (int i = 0; i < list.Length; ++i)
            list[i] = rnd.Next();

        // sort table
        sorter.Sort(list);

        // verify
        for (int i = 0; i < list.Length - 1; ++i)
        {
            Assert.IsTrue(sorter.Comparer.Compare(list[i], list[i + 1]) <= 0,
                "Element {0} ({1}) is strictly greather that {1} ({2})",
                i, list[i], i + 1, list[i + 1]
            );
        }
    }

    [Test]
    [ExpectedArgumentNullException]
    public void SortNulList(ISorter sorter)
    {
        sorter.Sort(null);
    }

    [Test]
    public void SortEmptyList(ISorter sorter)
    {
        CreateSortAndCheckSorted(sorter, 0);
    }
    [Test]
    public void SortListWithOneElement(ISorter sorter)
    {
        CreateSortAndCheckSorted(sorter, 1);
    }
    [Test]
    public void SortListWithTwoElements(ISorter sorter)
    {
        CreateSortAndCheckSorted(sorter, 2);
    }
    [Test]
    public void SortListOfSize100(ISorter sorter)
    {
        CreateSortAndCheckSorted(sorter, 100);
    }
}

Now that we have defined the fixture, we need to feed it with ISorter instances. Usually, you would need to write a factory class that would create the instance, but this task was too boring not to be automated. What I want is a way to tell MbUnit to look for all the classes in NProf that implemented ISorter that apply the fixture to it...

Implementing a custom factory attribute:

The TypeFixture fixture looks for attributes that derive from ProviderFixtureDecoratorPatternAttribute and invoke their GetRun method to gather the IRun instance. Therefore, we "just" need to inherit from this attribute and implement the behavior we want:

[AttributeUsage(AttributeTargets.Class,AllowMultiple =true,Inherited =true)]
public class AssemblyProviderFactoryAttribute : ProviderFixtureDecoratorPatternAttribute
{
    private Type assemblyType; // type contained in the assembly to test
    private Type targetType; // the tested type
    public AssemblyProviderFactoryAttribute(Type assemblyType, Type targetType)
    {
        ...
        this.assemblyType = assemblyType;
        this.targetType = targetType;
    }
    public Assembly Assembly
    {
        get { return this.assemblyType.Assembly;}
    }
    public Type TargetType
    {
    get { return this.targetType;}
    }
    public override IRun GetRun(Type decoratedType)
    {
        throw new NotImplementedException();
    }
}

The GetRun still needs to be implemented. The task for the returned Run object is to explore the tested Assembly (assemblyType.Assembly) for types  compatible with targetType. MbUnit provides an abstract base class, Run, for implementing IRun:

public class AssemblyProviderRun : Run
{
    private AssemblyProviderFactoryAttribute attribute;
    public AssemblyProviderRun(AssemblyProviderFactoryAttribute attribute)
    {
        this.attribute = attribute;
    }

    public override void Reflect(RunInvokerTree tree, RunInvokerVertex parent, Type t)
    {
        foreach (Type type in this.attribute.Assembly.GetExportedTypes())
        {
            if (!type.IsClass) // interrested in class only,
                continue;
            if (type.IsAbstract) // abstract class cannot be created
                continue;
            if (!this.attribute.TargetType.IsAssignableFrom(type)) // must be assignable to TargetType
                continue;
            // trying to get the default constructor
            ConstructorInfo ci = type.GetConstructor(Type.EmptyTypes);
            if (ci == null)// no default constructor
                continue;

            ActivatorRunInvoker invoker = new ActivatorRunInvoker(this, type);
            tree.AddChild(parent, invoker);
        }
    }
}

ActivatorRunInvoker is a IRunInvoker implementation that takes a type, creates an new instance of it and feeds it to the arguments of the next invoker. We implement this invoker by inheriting from the abstract base class RunInvoker:

public class ActivatorRunInvoker : RunInvoker
{
   private Type targetType;
   public ActivatorRunInvoker(IRun run, Type targetType)
   :base(run)
   {
      this.targetType = targetType;
   }
   public override string Name
   {
   get {return this.targetType.Name;}
   }
   public override object Execute(object o, System.Collections.IList args)
   {
      Object target = Activator.CreateInstance(targetType);
      args.Add(target);
      return null;
   }
}

That's the end of the journey. The attribute is ready to be used to tag the SorterTest fixture:

[TypeFixture(typeof(ISorter))]
[AssemblyProviderFactory(typeof(ISorter),typeof(ISorter))]
public class SorterTest
{
   ...
}

Let's launch those test and see what happens. I like having my Test assemblies "self-executable" so I make it a Console application and the AutorRunner in the Main method. The Text report spits out:

Tests run: 75, Failures: 0, Not run: 0

This makes sense because there are 15 ISorter implementation times 5 tests applied to them. Of course, I would like to have more information, so I generate the Html report:

Here we see clearly which class is being tested and so, our new attribute is working perfectly without recompiling MbUnit!

Testing NSort.Generics

In order to test the generic sort algorithm, we need to take the following steps:

  1. adapt the SorterTest fixture for the ISorter<int> interface (I choose to test int only),
  2. adapt the AssemblyProviderRun to support generics

The task 1 is trivial and is similar to the ISorter to ISorter<T> transformation. The second task is more technical because we need to check that types are generic and handle them differently. To cut the story short, here is the method that iterates over generic types:

private void ReflectGenericTypes(RunInvokerTree tree, RunInvokerVertex parent)
{
    Type[] args = this.attribute.TargetType.GetGenericArguments();
    foreach (Type type in this.attribute.Assembly.GetExportedTypes())
    {
        if (!type.IsClass)
            continue;
        if (type.IsAbstract)
            continue;
        if (!type.HasGenericArguments) // true if type is a generic
            continue;

        // get the generic type definition
        Type genericType = type.GetGenericTypeDefinition();
        // bind types
        Type gtype = genericType.BindGenericParameters(args);
        // check if assignable
        if (!this.attribute.TargetType.IsAssignableFrom(gtype))
            continue;
        // create new invoker
        AddInvoker(tree, parent, gtype);
    }
}

It took me a while to fix that up but I finally got though. Now, the GenericSorterTest looked as follows and I was ready to hit the "Run" button:

[TypeFixture(typeof(ISorter<int>))]
[AssemblyProviderFactory(typeof(ISorter), typeof(ISorter<int>))]
public class GenericSorterTest
{...

The text report of the tests returned 150 tests, which was logical and they all passed (after some bug fixing). And I'm happy.

The method of testing I showed here worked well because of the nature of the problem to test: the ISorter class are all well separated and do not need mocking and so on.  One of the nicest thing about this is that if you write new ISorter implementations, you do not need to modify your fixutre, they will be automatically tested by MbUnit.

 

posted on Thursday, August 19, 2004 12:27:00 AM UTC  #    Comments [0]

The date finally came through. I will present a workshop on testing on September 9 for the BENUG (Benelux .Net User Group) association on Testing. This will be a "exercise" oriented workshop.

posted on Wednesday, August 18, 2004 9:13:00 AM UTC  #    Comments [6]
 Tuesday, August 17, 2004

Eric Gunnerson has recently blog about a Trebuchet (that he offered to his wife?). This remembered me of the one I built, with my friends, a few years ago.

The photo below shows the "implementation" of the Trebuchet and some facts about it:

  • the rotating pole was 7m long and composed of two wooden poles of 15cm of diameter
  • the rotation axis was made of a steel stick of 26mm of diameter. It was at 3.5 m of the ground. Note because the construction was not perfectly symmetric, it got twisted.
  • the "counter"-weight was about 100kg altough not measured,
  • the structure was composed of 10 wooden poles buried 50cm into ground.
  • the projectile was a piece of wood, 50cm long, 15cm diameter, it got throwed at around 60 meter althought the trajectory was far from behing optimal (too vertical).

posted on Wednesday, August 18, 2004 12:55:00 AM UTC  #    Comments [4]

At first sight it may seem that Singleton and Unit Testing are not compatible since you cannot ensure the separation between the tests. We could solve the problem by instanciating a new "Singleton" instance for each test and apply test on this "local" instance. The problem is that a well implemented Singleton is sealed and it's constructor are private (see Singleton Pattern here) and thus you cannot create this instance.

using System;

public sealed class Singleton
{
   ...
   private Singleton() {}

   public static Singleton Instance
   {
      get { ...}
   }
}

Reflection to the rescue

An elegant way for bypassing the "creation problem" is to use Reflection to get the private constructor (ConstructorInfo) and then use this constructor info to create a new "Singleton" instance.

using System.Reflection;

[TestFixture]
public class SingletonTest
{
    private Singleton target = null;
    [SetUp]
    public void SetUp()
    {
        ConstructorInfo ci =
            typeof(Singleton).GetConstructor(
                BindingFlags.Instance |
                BindingFlags.NonPublic,
                null,
                Type.EmptyTypes,
                null
                );
         Assert.IsNotNull(ci);
         this.target = (Singleton)ci.Invoke(null);
    }
}

Now we have a new instance of the Singleton for each test and we are happy.

posted on Wednesday, August 18, 2004 12:14:00 AM UTC  #    Comments [18]
 Friday, August 13, 2004

Jamie Cansdale has found a hidden gem in .Net 2.0: the much awaited InternalsVisibleTo attribute. This attribute let's you specify the name of an assembly that can view the internals... That's perfect for testing internal classes.

http://weblogs.asp.net/nunitaddin/archive/2004/08/13/214130.aspx

posted on Saturday, August 14, 2004 3:51:00 AM UTC  #    Comments [2]

QuickGraph now contains a condensation graph and transitive closure algorithm which were ported from the BGL by Rohit Gadogkar. Here's a recap. of the definitions of those type of graphs (takened from the Boost Graph Library reference)

A condensation graph is a a graph CG=(CV,CE) based on the graph G=(V,E) where each vertex in CV corresponds to a strongly connected component in G and edge (u,v) is in CE if and only if there exists an edge in E connecting any of the vertices in the component of u to any of the vertices in the component of v.

The transitive closure of a graph G = (V,E) is a graph TG = (V,TE) such that TE contains an edge (u,v) if and only if G contains a path (of at least one edge) from u to v.

Condenstation Graph

Assume that you have built a graph g using QuickGraph:

IVertexListGraph g = ...;

The algorithm responsible for the condensation graph creation is QuickGraph.Algorithms.CondensationGraphAlgorithm and takes the graph to condensate as parameter:

CondensationGraphAlgorithm condensator = new CondensationGraphAlgorithm(g);

In order to compute the condensation, we need to provide a mutable graph, for example a AdjacencyGraph, and call the Create method:

AdjacencyGraph cg = new AdjacencyGraph(true);
condensator.Create(cg);

At this moment, cg contains the condensation graph of g. The only problem here, is that we have not stored the information about which vertex from V was associated to a vertex in CV. In order to do that, we need to add some event handlers to the algorithm. 

This algorithm defines one event, InitCondensationGraphVertex, which is trigerred for each new vertex of the condenstation graph CG. Remember that this vertex (in CV) "contains" all the vertices of a component in the original graph. In this example, we will create a condensation graph that creates NamedVertex, which has the Name property, and we concatenate the names of the condensed vertices to make it the name of the condensated vertex:

// using NamedVertexProvider in order 
// to create NamedVertex instances
AdjacencyGraph cg = new AdjacencyGraph(
    new NamedVertexProvider(),
    new EdgeProvider(),
    true);

// attaching event handler
condensation.InitCondensationGraphVertex+=
    new CondensationGraphVertexEventHandler(condensator_InitCondensationGraphVertex);
// computing
condensator.Create(cg);

...
// Event handler
void condensation_InitCondensationGraphVertex(
    object sender, 
    CondensationGraphVertexEventArgs arg)
{
    NamedVertex cv = (NamedVertex)arg.CondensationGraphVertex;
    
    StringWriter sw = new StringWriter();
    foreach (IVertex v in arg.StronglyConnectedVertices)
    {
        sw.Write("{0}\\n", v);
    }
    v.Name = sw.ToString();
}

That's it... Here are two images of graphs. The left image shows a state graph that is not strongly connected and the right image shows the condensation graph of it.

posted on Saturday, August 14, 2004 12:44:00 AM UTC  #    Comments [0]
 Tuesday, August 10, 2004

Iterators is one of the big, expected, new feature of C# 2.0. It should revolutionize the way you define your collection. Since iteration is fun, people have started to do some interresting and smart things with tem. So I decided, it was time I started playing with them. I'll assume that you have a basic knowledge of the new Generic syntax.

Background: STL algorithms

There was some ancient time where I was still a C++ user. At that time, I was using the Standard Template Library (STL) , a famous C++ library. This library makes an extensive use of template and what is called "generic programming". This library features containers, iterators and algorithms and was the basis of the work I have done in this post.

An introductrory example: filtered enumeration

Let's start with a simple, and yet, usefull iterator: we would like to create a filtered enumeration with regards to a given predicate. To do so, we first define the predicate interface (this interface defines a functor, funtion-like object)

public interface IPredicate<T>
{
    bool Invoke(T item);
}

And then, we can define the Select enumerator which returns the item of the predicate is true:

static public IEnumerable<T> Select<T>(IEnumerable<T> collection, IPredicate<T> pred)
{
    foreach(T item in collection)
    {
        if (pred.Invoke(item))
            yield return item;
    }
}

Let's apply this iterator to an example. Consider a list of string containing some names and a predicate that checks if a string starts with a particular string:

public class StartsWithPredicate : IPredicate<String>
{
    private string start;
    public StartsWithPredicate(string start)
    {
        this.start=start;
    }
    public bool Invoke(string item)
    {
        if (item==null)
            return false;
        return item.StartsWith(start);
    }
}

...
List<String> names = new List<String>();
names.Add("Marc");
names.Add("Jonathan");
names.Add("Julia");

Iterating and displaying the names that start with "J" is done using the Select method:

foreach(string name in Select(names, new StartWithPredicate("J")))
{ Console.WriteLine(name);}

-- output
Jonathan
Julia

This snippet is alreay nice but it is still quite verbose since we have to define a class before using it as a filter. It would be nice to use another new feature, namely anonymous methods, to define filters:

foreach(string name in Select(names, delegate(string s){ return s.StartsWith("J");}))
{ Console.WriteLine(name);}

In order to acheive this, we need to do 3 things: 1) declare a delegate that matches the signature of IPredicate.Invoke method, 2) create a IPredicate instance that wraps up a call to such delegate, 3) overload Select to take a delegate as argument:

// 1
public delegate bool PredicateDelegate(T item);

// 2
public class PredicateDelegateInvoker<T> : IPredicate<T>
{
   private PredicateDelegate<T> del;
   public PredicateDelegateInvoker(PredicateDelegate<T> del)
   {
      this.del = del;
   }
   public bool Invoke(T item)
   {
      return del(item);
   }
}

// 3
public IEnumerable<T> Select<T>(IEnumerable<T> collection, PredicateDelegate<T> pred)
{
   return Select(collection, new PredicateDelegateInvoker<T>(pred));
}

Now, we can use anonymous methods as shown above to filter any enumeration. In the following, I will define a framework to define a variety of enumerator, like in the good old days of STL.

Enumerator framework

1 Functors

First, we need to define some functor interfaces. There are two main families. Transformers take T instances as arguments and return a R instance. A transformer can accept one or two arguments (Binary). This interface represents the conversion T -> R:

public interface ITransformer<T,R>T
{
    R Invoke(T item);
}
public interface IBinaryTransformer<T,R>
{
    R Invoke(T left, T right);
}

Predicates are a specialized kind of transformer which returns bool:

public interface IPredicate<T> : ITransformer<T,bool>
{}

public interface IBinaryPredicate<T> : IBinaryTransformer<T,bool>
{}

There are a number of pre-built predicate that we can implement: And, Or, Not, etc... I will show how the And predicate, which applies the boolean and operator to the result of two IPredicate is implemented:

public class AndPredicate<T> : IPredicate<T>
{
    private IPredicate<T> leftPred;
    private IPredicate<T> rightPred;
    public AndPredicate(IPredicate<T> leftPred, IPredicate<T> rightPred)
    {
        this.leftPred = leftPred;
        this.rightPred = rightPred;
    }
    public bool Invoke(T value)
    {
        return leftPred.Invoke(value) && rightPred.Invoke(value);
    }
}

2 Delegates and delegate wrappers

As in the introductory example, we define 4 delegates and their functor wrapper for each functor interface:

public delegate R TransformerDelegate<T,R>(T item);
public delegate R BinaryTransformerDelegate<T,R>(T left, T right);
public delegate bool PredicateDelegate<T>(T item);
public delegate bool BinaryPredicateDelegate<T>(T left, T right);

+ wrappers

The wrappers are proxies to the delegate invokation.

3 Algorithms

Now that we are equipped with functors, we can start having fun and defining algorithms/enumerators.

3.1 Select

The Select enumerator (filtered enumeration) was already defined in the example, so we can skip it.

3.2 Transformation

The Transform enumerator applies a ITransformer to each item and returns the result:

static public IEnumerable<R> Transform<T,R>(
    IEnumerable<T> collection,
    ITransformer<T,R> transformer)
{
    foreach (T item in collection)
    {
        yield return transformer.Invoke(item);
    }
}

We also take care of writing the overload that handles anonymous methods. For example, we can use this method to convert all the strings in the names collection to upper case:

foreach(string upperCaseName in Transform<string,string>(names,delegate(string s){return s.ToUpper();}))
{Console.WriteLine(name);}

-- output
MARC
JONATHAN
JULIA

Note that in this case,  we need to specify the T,R parameters because the compiler cannot resolve them.

3.3 Others...

The list of possible enumeration is quite long, so I'll stop here for today.

Download the bits

You can download the bits from this example at www.dotnetwiki.org (look for Iterate project in the download section).

 

posted on Wednesday, August 11, 2004 4:24:00 AM UTC  #    Comments [4]
 Monday, August 09, 2004

Updated the "sick" Reflector.Graph release.

  • Compiled against .Net 1.0 and .Net 1.1 (1.0 version does not have the TreeMap)
  • Various bugs fixed

Download at www.dotnetwiki.org

posted on Monday, August 09, 2004 8:45:00 PM UTC  #    Comments [0]
 Saturday, August 07, 2004

There's one new cool feature of the VS2005 that has not received much attention, CodeSnippets. With CodeSnippets, you can write code template in minutes that are directly integrated into the intellisense of VS. That's great.... the sad point is that they are totally undocumented :(

CodeSnippets

Here's a small description of CodeSnippets in VS Express and here's a better tutorial but it is in french.

CodeSnippets for MbUnit

MbUnit has now CodeSnippets to create fixture, test methods, etc... all kinds of snippets that you write again and again. To set up the snippets, you need VS2005 and to download the MbUnit Code Snippets from www.dotnetwiki.org (they will be integrated in the next release).

Unzip the file in a directory and go to the menu Tools -> Code Snippets Manager... In the dialog, push Add and choose the directory where you have unzipped the snippets. The dialog window should look as follows. Clicking on each snippet will give you a short description, the shortcut to execute it, etc...

TestFixture snippet

In you need to create a new fixture, the TestFixture snippet is now here to help. Create a new blank C# file and write te. Intellisense window should pop out and now you will see new items: test, testfixture, etc...

 Go to the textfixture item and double "tab" it. The editor will execute the template and expand the item into an empty TestFixture class definition as follows:

 

You can also access the MbUnit expansion fixture by looking in the context menu, Intellisense -> Expand Item... -> MbUnitExpansion

posted on Sunday, August 08, 2004 3:52:00 AM UTC