Monday, May 10, 2004

I have just finished a CodeSmith template that "automatically" generates empty tests case for a given type.

This template does a little bit more than just enumerating the available methods: it explores MSIL using RAIL and creates test case following the given rules:

  • if the IL instruction is a conditional branch, create a test case for both possibilities (true/false case)
  • if the IL instruction throws, create a test case expecting the exception,
  • if the IL instruction is returning, create a test case that checks the returned value

Those rules are quite simplistic but can already generate a "huge" number of test case automatically. Each test case comes with a piece of documentation and the method IL code where the target instruction has been set in bold. Currently, the documentation contains IL code, but in the future it would be nicer to output real code, using Reflector for example.

In order to make the template work, you need to put RAIL assemblies and the AssemblyHelper assembly in the CodeSmith directory.

Here's a quick example. The method:

public void ABitMoreComplext(bool goForIt)
{
    if (goForIt)
        throw new Exception();
    else
        Console.Write("did not throw");
        // other instruction 
    Console.Write("hello");
}

and the resulting output in the generated class:

        
        /// <summary>
        /// Tests ABitMoreComplext method when condition is executed as true
        /// (see remarks).
        /// </summary>
        /// <remark>
        /// <para>
        /// Not implemented
        /// </para>
        /// <code>
        /// ldarg.1
        /// <b>brfalse.s</b>
        /// newobj
        /// throw
        /// ldstr
        /// call
        /// ldstr
        /// call
        /// ret
        /// </code>
        /// </remarks>
        [Test]
        [Ignore]
        public void ABitMoreComplextIfTrue1()
        {
            throw new NotImplementedException();
        }
        /// <summary>
        /// Tests ABitMoreComplext method when condition is executed as true
        /// (see remarks).
        /// </summary>
        /// <remark>
        /// <para>
        /// Not implemented
        /// </para>
        /// <code>
        /// ldarg.1
        /// <b>brfalse.s</b>
        /// newobj
        /// throw
        /// ldstr
        /// call
        /// ldstr
        /// call
        /// ret
        /// </code>
        /// </remarks>
        [Test]
        [Ignore]
        public void ABitMoreComplextIfFalse1()
        {
            throw new NotImplementedException();
        }
        /// <summary>
        /// Tests that the ABitMoreComplext method throws (see remarks)
        /// </summary>
        /// <remark>
        /// <para>
        /// Not implemented
        /// </para>
        /// <code>
        /// ldarg.1
        /// brfalse.s
        /// newobj
        /// <b>throw</b>
        /// ldstr
        /// call
        /// ldstr
        /// call
        /// ret
        /// </code>
        /// </remarks>
        [Test]
        [Ignore]
        [ExpectedException(typeof(Exception))]
        public void ABitMoreComplextThrow3()
        {
            // don't forget to update the exception type.
            throw new NotImplementedException();
        }

ps: The template is called TestFixture and is in the Templates directory in the MbUnit CVS.

 

posted on Tuesday, May 11, 2004 4:25:00 AM UTC  #    Comments [13]
Tracked by:
"free texas poker" (online) [Trackback]
Monday, June 06, 2005 6:04:21 PM UTC
I have to drop RAIL to use ILReader.
Jonathan de Halleux
Monday, June 06, 2005 6:04:21 PM UTC
Really nice. You should make this a Reflector add-in.
Alexander
Monday, June 06, 2005 6:04:22 PM UTC
That's exactly what I'm thinking about. I just realized that renaiming reflector to reflector.dll, you can link it with your project.
Jonathan de Halleux
Monday, June 06, 2005 6:04:23 PM UTC
It would be great if we could automatically generate the class that we are testing from the test code i.e. write a test harness that describles the functionallity of the class via the tests and the generator automatically generates a test class that adheres to this specification...
Punisher
Monday, June 06, 2005 6:04:23 PM UTC
Punisher: That is TDD pushed to the extreme. I don't see any &quot;easy&quot; implementation of this.
<br>
<br>Another solution would be to describe the class functionalities in xml format so that you can render it to the class and the test class.
Jonathan de Halleux
Monday, June 06, 2005 6:04:24 PM UTC
Thats kind of what I had in mind...
<br>
<br>Time to get our thinking caps on!
Punisher
Monday, June 06, 2005 6:04:24 PM UTC
I'm probably missing something but I've not been able to get this template to work. I've copied Rail Assemblies and AssemblyHelper dll into the folder where I've setup the template and I've copied the <a title="QuickGraph, 100% C# directed graph library" href="http://mbunit.tigris.org" target="_blank">QuickGraph</a> assemblies into there as well. Problem is the latest <a title="MbUnit, Generating Unit Testing and Model Based Testing Framework for .NET Framework" href="http://mbunit.tigris.org" target="_blank">MbUnit</a> build (downloaded from <a title="TestDriven.NET" href="http://www.testdriven.net" target="_blank">TestDriven.NET</a>) doesn't contain a number of the <a title="QuickGraph, 100% C# directed graph library" href="http://mbunit.tigris.org" target="_blank">QuickGraph</a> assemblies. I had a look in the <a title="MbUnit, Generating Unit Testing and Model Based Testing Framework for .NET Framework" href="http://mbunit.tigris.org" target="_blank">MbUnit</a> cvs repository and sure enough the source code for the <a title="QuickGraph, 100% C# directed graph library" href="http://mbunit.tigris.org" target="_blank">QuickGraph</a> assemblies is there. Checked them out and tried to build them, but you need .NET framework 2 as MSBUILD is being used. Downloaded and installed that, no joy. Installed mutantbuild and changed the nant script to use mutant build instead of MSBUILD. Still no joy. I noticed that a number of the assmeblies are signed and the the keys weren't in cvs, so removed the references to the keys from the assemblyinfo.cs files. Still no joy.
<br>
<br>I now consider myself to have hit a brick wall. Does someone have a zip file with all the relevant assemblies they can post somewhere for easy download?
<br>
<br>Thanks in advance.
Neil Fillingham
Monday, June 06, 2005 6:04:25 PM UTC
Andrew Stopford's Weblog
Monday, June 06, 2005 6:04:25 PM UTC
Andrew Stopford's Weblog
Monday, June 06, 2005 6:04:26 PM UTC
I'm lost: can someone post a simple explanation of how to download and what to do with this, or if it moved to &quot;Reflector&quot; how to download <a title="Reflector" href="http://www.aisto.com/roeder/dotnet/" target="_blank">Reflector</a> and use this?
<br>
<br>We (the developers) need one long simple download and install guide with numbers: 1. Download this from here. 2. Download that from there. 3. Do this. 4. Double click on that.
<br>
<br>Thanks! Moshe
Pashute: Extreme Programming Israel
Monday, June 06, 2005 6:04:27 PM UTC
There's no free lunch.
<br>
<br>1. Load <a title="Reflector" href="http://www.aisto.com/roeder/dotnet/" target="_blank">Reflector</a>.Framework.dll (see <a target="_new" href="http://projectdistributor.net">http://projectdistributor.net</a> )
<br>2. Right clik on a class and choose &quot;Code Generator&quot;
<br>3. Select &quot;Unit test case generator&quot; on combo box.
<br>
Jonathan de Halleux
Monday, June 06, 2005 6:04:28 PM UTC
The addin fails to load:
<br>System.Reflection.ReflectionTypeLoadException: One or more of the types in the assembly unable to load.
<br> at System.Reflection.Module.GetTypesInternal(StackCrawlMark&amp; stackMark)
<br> at System.Reflection.Assembly.GetTypes()
<br> at _1._1(IServiceProvider )
<br> at _5._1(String )
<br>
Michael Friedgeim
Monday, June 06, 2005 6:04:28 PM UTC
Wow, superb!
<br>Thanks.
Diana
Comments are closed.