# Thursday, June 24, 2004

Little intro

Reflector is a (great) decompiler. The most obvious use of it is to load an assembly and recreate source code out of the IL it contains. The assembly itself was built from source code, so we have a mini "cycle of life". In a perfect world, the decompiled code and the orginal source code would be exactly the same (at least in the same language). So we can represent this cycle by the figure below.

Now,  let's introduce a new actor in this ecosystem. We want the decompiler to producec CodeDom source code that can be used to generate the original source code as shown in the figure below: we want to produce a CodeDom template that will generate that type.

Sample example

Let's illustrate all this with an example. Consider the following User class:

public class User
{
    private string name = "Marc";

    public String Name
    {
        get
        {
            return this.name;
        }
        set
        {
            this.name=value;
        }
    } 
    public override string ToString()
    {
        return this.name;
    }
}

We compile User and now, we want Reflector to generate a class that uses CodeDom (indeed Refly) to generate back the User class. This is the tricky part. I need to use Refly to generate code that uses Refly, so that would be building a code generator generator. The template for the User class will look like this:

public class UserTemplate 
{
    private Refly.CodeDom.ClassDeclaration _type;
    private Refly.CodeDom.FieldDeclaration _nameField;
    private Refly.CodeDom.PropertyDeclaration _nameProperty;
    private Refly.CodeDom.MethodDeclaration _toStringMethod;

    public UserTemplate(Refly.CodeDom.NamespaceDeclaration ns) 
    {
        // add User class to generator
        this._type = ns.AddClass("User");
        this.Declare();
    }

    public virtual void Declare() 
    {
        this.DeclareFields();
        this.DeclareMethods();
        this.DeclareProperties();
        this.DeclareEvents();
        this.DeclareInnerClasses();
    }

    public virtual void DeclareFields() 
    {
        // adding name field
        this._nameField = this._type.AddField(typeof(String), "name");
    }

    public virtual void DeclareMethods() 
    {
        this._toStringMethod = this._type.AddMethod("ToString");
        this.PopulateToStringMethod(this._toStringMethod);
    }

    public virtual void DeclareProperties() 
    {
        this._nameProperty = this._type.AddProperty(typeof(String), "Name");
        this.PopulateNameProperty(this._nameProperty);
    }
    ...
}

(The above template is still missing a big part: the generation of the statements and the expression) Now that we have a Refly ClassDeclaration, we can feed it to the CodeGenerator class:

NamespaceDeclaration ns = new NamespaceDeclaration("RoundRobin");
UserTemplate template = new UserTemplate(ns);
CodeGenerator gen =new CodeGenerator();
gen.GenerateCode("./Output",ns);

And the result of the execution of this program will be this piece of code:

namespace RoundRobin 
{
    using System;
    public class User 
    {
        private string _name;
        public virtual string Name 
        {
        }
        public virtual void ToString() 
        {
        }
    }
}

Not so bad for a start, we have gotten our User class back but there are still some big "holes" in it....

posted on Thursday, June 24, 2004 8:40:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [0]

Following the idea of displaying a class in a PropertyGrid, we can also take a look at how it serializes using the XmlSerializer (see if it serializes at all!). The task looks trivial at first sight:

  1. create an instance of the object,
  2. create a XmlSerializer that can serialize it,
  3. write the object to a XmlRichTextWriter to get colors :)

Of course, there's a gotcha here. You will likely get empty xml files all of the time because usually, the default constructor creates "empty" object. So what we want is to dress the object before serializing them. This is done in a variety of ways depending on the property type:

  • the property/field is a string, int, etc... you can set easily a value to it,
  • the property/field is tagged with XmlArrayAttribute,
    • the property/field is an array: get all the XmlArrayItem attribute, create an array and fill it with instance of those items,
    • the property/field is a collection: get the "Add(Object)" method, add instances of the items.

Of course, this is rather simplistic and may fail on forgotten situations.

Screenshot: ReportAssembly contains collection properties (namespaces, fixtures),  which would appear empty if not "dressed".

posted on Thursday, June 24, 2004 7:23:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [2]
# Wednesday, June 23, 2004

In this blog, I'll show how you can a implement a control to colorizes XML in a simple and effective way. I needed to display XML in a control, however, displaying it in "black and white" made it very depressing.  I have 3 objectives: simplicity, simplicity and simplicity.You can think of this as a traditional Microsoft interview enigma. So let's enumerate what we need and what we want (and what we want to avoid). 

What we want: colors!
What we don't want: manipulate the XML Dom, in other words: let's be lazy.
What we have: it's time to dig into the FCL classes and see what we could use 

  • RichTextBox: this control seems to be the perfect match for our "coloring" purposes,
  • XmlTextWriter: this class is a forward only, non-cached Xml writer. This class provides a rich set of method that output elements, attributes, etc...

Implementation

At this point, you must have an idea of the way we are going to acheive colors... the tools are here, we just need to inject the color at the right spot (like in a toothpaste, the color lines come right at the end). In fact, we have 3 tasks to acheive:

1) RichTextBoxWriter:

A TextWriter-derived class that writes to a RichTextBox. The writer keeps a current Color and Font that is apply to each chunk of string appended. This task is pretty straightforward, basically you need to overload 3 Write methods.

public class RichTextBoxWriter : TextWriter
{
    private RichTextBox textBox;
    private Color currentColor=Color.Black;
    private Font currentFont = new Font("Tahoma",8.25f);

    ...

    public override void Write(char value)
    {
        this.TextBox.SelectedText = value.ToString(this.FormatProvider);
    }
    public override void Write(char[] buffer)
    {
        if (buffer!=null)
        {
            this.TextBox.SelectedText = new String(buffer).ToString(this.FormatProvider);
            // if buffer = '\n\r', font and color is reseted
            this.TextBox.SelectionColor=this.CurrentColor;
            this.TextBox.SelectionFont=this.CurrentFont;
        }
    }
    public override void Write(string value)
    {
        this.TextBox.SelectedText = value.ToString(this.FormatProvider);
    }
}

Wow, that was short. Now to point 2.

2) XmlRichTextWriter:

This class, which inherits from XmlTextWriter, injects color on some "startegic" methods. In order to store the colors, we create XmlRichTextBox, which inherits from RichTextBox and contains a rich set of customizable colors and fonts for the Xml rendering. This class will be used by the XmlRichTextBoxWriter:

public class XmlRichTextBoxWriter : XmlTextWriter
{
    private XmlRichTextBox theme; // contains all the colors and fonts

    public XmlRichTextBoxWriter(XmlRichTextBox tb)
    :base(tb.Writer)
    {
        this.theme=tb;
    }

    public override void WriteStartDocument()
    {
        // setting color of the selected text
        this.theme.Writer.CurrentColor = this.theme.StartColor;
        this.theme.Writer.CurrentFont = this.theme.StartFont;
        // output data
        base.WriteStartDocument();
    }
    ...
}

4) XmlRichTextBox:

Now, that we have our writer, we just need to wire them up in the XmlRichTextBox and we are done.

And finally, the result: on the left, a XmlRichTextBox, on the left, how it looks like in the PropertyGrid. We have colooors!

Download it now....

posted on Wednesday, June 23, 2004 9:08:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [6]

Joel Pobar speaks about the new features of Reflection.

ReflectionOnly seems to be very interresting...

posted on Wednesday, June 23, 2004 9:14:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [1]
# Tuesday, June 22, 2004

Ever wondered how you classes you look like in a PropertyGrid control, now it is possible directly from Reflector with my upcoming Reflector Addin:

Ps: As always, a big thanks to Jamie Cansdale (NUnitAddIn) for his help on this one.

posted on Tuesday, June 22, 2004 4:19:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [5]

XsdTidy is a refactoring tool to overcomes some silly limitations of the exceptional Xsd.exe  tool provided with the .NET framework. More specifically, XsdTidy addresses the following problems:

  • Name normalization: if your XSD schema is using lower case names or more generally non ".NET" normalized names, you will end up with types that will make the FxCop spit out hundreds of infractions,
  • Fields not properties: xsd.exe creates fields and does not "hide" them in properties which is bad OO design.
  • Fixed Array Sizes: xsd.exe handles multiple elements by creating an array. There is no problem when you are loading the data, but unfortunately this is not convenient if you want to populate a document since arrays do not support Add or Remove. XsdTidy provides strongly-typed collection that support Add, Remove, etc...
  • Default Constructor: Xsd.exe does not care about providing a default constructor that initializes the fields with the proper values. If the object structure is getting big, it becomes very difficult to properly initializes fields,
  • Serializable: Xsd.exe does not tag classes with serializable

Note that XsdTidy uses Refly for building the source code and is also maintained by Marcus Mac Innes.

What does XsdTidy fix ?

Name conversion

The .NET standards define specific naming convention for all types of data: arguments should be camel case, function names capitalized, etc... This is really helpful to keep the framework consistent. Tools like FxCop help us stay on the "normalized" side.

This problem is tackled the dumb way: given a dictionary of "common" words, the class NameConformer tries to split a name in separate words, after that it renders it to the needed convention. Of couse, this feature can be disabled.

FixedArraySize and "Multi" Strongly-Typed Collections

Arrays are replaced by inner strongly-type collections which are much more flexible to use. Moreover, array fields are created by default using their default constructor. This is to economize you the hassle of creating a collection before using it. If an array can support multiple object type, the generate collection will be "multi" strongly typed:

public class TestClass
{
    [XmlArray("values")]
    [XmlArrayItem("car",typeof(Car));
    [XmlArrayItem("car",typeof(Bike));
    public Object[] values;
}

becomes

public class TestClass
{
    private ValueCollection values = new ValueCollection();

    [XmlArray("values")]
    [XmlArrayItem("car",typeof(Car));
    [XmlArrayItem("car",typeof(Bike));
    public ValueCollection Values
    {
        get { return this.values;}
    }

    public class ValueCollection : CollectionBase
    {
        public void AddCar(Car car)
        {
            this.List.Add(car);
        }
        public void AddBike(Bike bike)
        {
            this.List.Add(bike);
        }
        ... // the implementation of the collection
    }
}

Properties

Fields are hidden in properties, which is more convenient to use. Moreover, collection fields do not have set property according to FxCop rule.

public class testclass
{
    [XmlAttribute("values")]
    public String values;
}

becomes:

public class TestClass
{
    private String values;

    [XmlAttribute("values")]
    public String Values
    {
        get
        {
            return this.values;
        }
        set
        {
            this.values = value;
        }
    }
}

Serializable

The output classes are tagged with the Serializable attribute to make them usable using Remoting.

XsdTidy history

I have first started to build XsdTidy using System.Reflection.Emit. It was a titanic job and very error prone. The difficulty of using Emit pushed me to use CodeDom which was also heavy to use. So finally, Refly was designed and XsdTidy became much easier to impement.

Download:

Download XsdTidy and Refly at http://blog.dotnetwiki.org/downloads/Refly.zip

Screenshot

posted on Tuesday, June 22, 2004 9:33:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [6]
# Monday, June 21, 2004

Reflector.Graph was becoming messy because of external dependencies so I decided to remove some parts of it (TypeGraph, TreeMap) in order to keep funcitonal.

Reflector version: tested for 4.0.6.0 and 4.0.7.0

Installation procedure:

  • Download the Reflector.Graph Binaries  and unzip it in the Reflector directory (it may be unzipped in a Release directory, copy the file aside to Reflector)
  • In Reflector, Tools -> Addin... -> Add -> Reflector.Graph.dll
  • That's it!

In this release, you do not need to create a Reflector.exe.config.

What's in Reflector.Graph:

  • MethodRank (Assembly),
  • TypeRank (Assembly),
  • Unit Test Generator (Type),
  • Mock Geneator (Type),
  • IL Graph (Method),
  • AssemblyGraph (Tools)

What's not in Reflector.Graph:

  • TypeGraph will be release separately because it needs to be compiled agains .Net v1.1
  • TypeTreeMap will be released separately because it has a dependency on the Microsoft TreeMap

Bugs and suggestions:

 

posted on Monday, June 21, 2004 8:56:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [2]
# Sunday, June 20, 2004

The Database Populator Framework  follows the same idea of DbMonster (or at least, was partially inspired from it). The purpose of the framework is to provide a flexible (and smart) set of random data generators to populate your database.

Why ?

If you plan to do database testing, the solution are not much: use transaction, create/kill database after each unit test, clean up after each test or use the new technique using Enterprise Services.

The objective of DPF is not to test the database but provide "food" for your unit test. By generating data randomly (satisfying the constraint), you do not need to clean up and you will avoid test clashing on each other. Moreover, you can also use the DPF to test your database under load.

Random but no dumb

In order to generate data for testing database, you need to take into accounts the constraints on the columns and between the tables. By providing a DataSet representing you database, the framework analyses it and create a set of generators. The data generated satisfies all the constraint. Of course, each row generator can be customized for your own needs, but in general, the information contained in the dataset is enough.

The ideas

The idea are rather simple:

  • Each DataColumn has its corresponding IDataGenerator instance,
  • Each DataTable has its corresponding ITablePopulator instance,
  • Each UniqueConstraint has its corresponding IUniqueValidator instance which takes care of ensuring the all unique constraint are validated,
  • Each ForeignKeyConstraint has its corresponding IForeignKeyConstraint instance which will fetch valid foreign key values

The generation of a new row is made inside a ITablePopulator as follows:

  1. Create a row,
  2. Call each IDataGenerator and fill the row,
  3. For each foreign key, get relevant values,
  4. Verify unique constraints, if violated go to 1,
  5. return row

The first bits

The first bits of the framework is located in the MbUnit.Framework.Data namespace. Althout the framework still needs work, the first example is running and is quite promising.

I have built a simple database containing User - Order - Product - OrderProduct:

this.dataSet=new DataSet();

DataTable users=dataSet.Tables.Add("Users");
DataColumn userID = users.Columns.Add("UserID",typeof(int));
DataColumn userName=Users.Columns.Add("UserName",typeof(string));
DataColumn userName.AllowDBNull=false;

DataTable orders=dataSet.Tables.Add("Orders");
DataColumn orderID=orders.Columns.Add("OrderID",typeof(int));
DataColumn orderDate = orders.Columns.Add("OrderDate",typeof(DateTime));
DataColumn oUserID = orders.Columns.Add("UserID",typeof(int));

DataTable products=dataSet.Tables.Add("Products");
DataColumn productID=products.Columns.Add("ProductID",typeof(int));
DataColumn productName = products.Columns.Add("ProductName",typeof(string));
DataColumn productPrice = products.Columns.Add("ProductPrice",typeof(decimal));

DataTable orderProducts=dataSet.Tables.Add("OrderProducts");
DataColumn opOrderID=orderProducts.Columns.Add("OrderID",typeof(int));
DataColumn opProductID=orderProducts.Columns.Add("ProductID",typeof(int));
DataColumn quantity=orderProducts.Columns.Add("Quantity",typeof(int));
// pks
users.Constraints.Add("PK_Users",userID,true);
orders.Constraints.Add("PK_Orders",orderID,true);
products.Constraints.Add("PK_Products",productID,true);
orderProducts.Constraints.Add("PK_OrderProducts",
new DataColumn[]{ opOrderID, opProductID}
,true);
// fks
orders.Constraints.Add("FK_Orders_Users",userID,oUserID);
orderProducts.Constraints.Add("FK_OrderProducts_Orders",orderID,opOrderID);
orderProducts.Constraints.Add("FK_OrderProducts_Products",productID,opProductID);

This database is easily populated using the DPF:

// creating populator
IDatabasePopulator pop=new DatabasePopulator();
// anaylising internal structure
this.pop.Populate(this.db.DataSet);

After that, you can customize the behavior of every data generator. Once this is done, you are ready to "feed" your database:

//getting the users table populator
ITablePopulator userPop = pop.Tables[users];
// adding new row
users.Rows.Add( userPop.Generate() );
posted on Sunday, June 20, 2004 11:03:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [8]

Refly is the library that I have developped to make CodeDom usable. (See my CodeProject article here, or the classic Hello World example). Note that Refly comes with XsdTidy, an application that "pretify" the output of the Xml.exe tool.

Download Now

 

posted on Sunday, June 20, 2004 10:17:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [3]

If you are doing database testing, you should read this great post by Roy:

Simplified Database Unit testing using Enterprise Services

posted on Sunday, June 20, 2004 8:31:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [1]
# Friday, June 18, 2004

The Production Grammar Framework is finally fully integrated into MbUnit as a new Fixture (GrammarFixture).

An example:

I'm currently building a new framework for generating "relevant" data for randomly populating database (similar ideas to DbMonster). As you may have noticed, I don't practive dogfood much so I decided to start now. I'm writing the test code along with the framework (of course, I'm testing myself here so it's a bit twisted). Anyway, I had to test the following collection:

public interface IDataGeneratorCollection : ICollection
{
    IDataGenerator this[DataColumn column]{get;}
    IDataGenerator this[String columnName]{get;}
    void Add(IDataGenerator dataGenerator);
    void Remove(IDataGenerator dataGenerator);
    void Remove(DataColumn column);
    void Remove(String columnName);
    bool Contains(IDataGenerator dataGenerator);
    bool Contains(DataColumn column);
    bool Contains(String columnName);
    void Clear();
}

At first, I decided to write a TypeFixture and produce a bunch of unit tests for that, but I quickly got bored, so I turned myself to production grammars which are much more fun. The grammar for this collection can be summarized as follows (see the stack example):

-- dg is a IDataGenerator
add := Add(dg)
remove := Remove(dg)
contains := Contains(dg)

guardedRemove := guard(InvalidOperatoinException, remove)

empty := add,guardedRemove,contains
nonEmpty := add,remove,contains

startRule := if(collection.IsEmpty) { empty } else { nonEmpty }

MbUnit Attributes

Now that we have built a grammar we need to tell MbUnit to load the grammar it feed it with seeds:  GrammarFixtureAttribute describes a production grammar fixture, GrammarAttribute describes a method that return a IGrammar instance, SeedAttribute returns an object that is feeded into the production constructor.

[GrammarFixture]
public class DataGeneratorCollectionGrammar : Grammar
{
    ... // rules creation etc...

    [Grammar]
    public Grammar This()
    {
        return this;
    }
    [Seed]
    public int Seed10()
    {
        return 10;
    }
    [Seed]
    public int Seed20()
    {
        return 20;
    }
    [Seed]
    public int Seed50()
    {
        return 50;
    }
    [Seed]
    public int Seed200()
    {
        return 200;
    }
}

Screenshot: the grammar test case + the console output

posted on Friday, June 18, 2004 3:51:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [3]

Aaron has written a "Hello World" sample using Refly.

Refly is a helper wrapper around the CodeDom namespace that makes it much less "noisy". Refly is used in the Unit Test Generator Addin. If you are using CodeDom regularly, maybe you should have a look at this.

posted on Friday, June 18, 2004 1:14:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [2]

This is a quick recap on the available features of MbUnit 2.15.1:

Fixtures:

Fixture type define how MbUnit is going to explore a class and build the test cases.

Fixture can be organized in a number of was. Automatically, they are organized by Namespace and Authors, but you can also tag a http://blog.dotnetwiki.org/archive/2004/05/29/277.aspx, and put them into multiple categories

Decorators:

Decorators are used on the "method" level to modify the behavior of a given test. Decorators can be chained to combine their effect.

Assertion classes

Aside from the classic Assert class, there are a number of specialized assertion classes:

 

posted on Friday, June 18, 2004 6:39:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [5]
# Thursday, June 17, 2004

I'm preparing a little Reflector Addin to be able to load and execute MbUnit fixtures inside Reflector. Since the fixture tree contains all the functionalities, it was just a matter of injecting it into Reflector.

Screenshot:

The source code is so short that I'm also publishing (note that I'm using the helper classes described here)

using System;
using System.Windows.Forms;
using Reflector.CodeModel;
using MbUnit.Forms;
namespace Reflector.Graph.Faulty
{ 
    // Reflector package
    public class MbUnitPackage : BasePackage
    {
        [ReflectorWindow("MbUnit")]
        [ReflectorCommandBar(CommandBarTarget.Assembly)]
        private MbUnitWindow MbUnit=new MbUnitWindow(); 
    }

    // Tree View
    public class MbUnitWindow : ReflectorTreeView
    {
        private ReflectorServices services =null;
        public MbUnitWindow()
        {
            this.Dock =DockStyle.Fill;
        }
        // needed to get reflector current element
        public ReflectorServices Services
        {
            get
            {
                return this.services;
            }
            set
            {
                if (this.services!=null)
                {
                    this.services.AssemblyBrowser.ActiveItemChanged-=new EventHandler(this.activeItem_Changed); 
                }
                this.services=value;
                if (this.services!=null)
                {
                    this.services.AssemblyBrowser.ActiveItemChanged+=new EventHandler(this.activeItem_Changed); 
                }
            }
        }
        private void activeItem_Changed(Object sender, EventArgs args)
        {
            this.RemoveAssemblies();
            IAssembly assembly = this.Services.ActiveAssembly;
            if (assembly!=null)
                this.Translate();
        }

        // populate tree with current assembly
        public void Translate()
        {
            IAssembly assembly =this.Services.ActiveAssembly;
            if (assembly==null)
                return;
            this.RemoveAssemblies();
            this.AddAssembly(assembly.Location);
            this.ThreadedPopulateTree();
        }
    }
}
posted on Thursday, June 17, 2004 11:06:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [3]
# Wednesday, June 16, 2004

MbUnit 2.1.5.1 Beta is ready for download.  See latest release page on this blog for the download links

Issues, feature requests, and other bugs: please do not post them on the blog. You have two options (please it's easier to track things)

We have a menu!

posted on Wednesday, June 16, 2004 5:53:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [17]

This is a preview of a new Addin in the next Reflector.Graph release: Type Tree Map.

A TreeMap is special 2D visualzation of a tree. Since, Reflector provides a tree (Assembly -> Module -> Namespace -> Type -> Members), I wondered if it was possible to visualize with a TreeMap. In .Net, there are 2 controls available that can render treemaps:

I decided to use the later for creating the addin.

Screenshot

Note (1): The nodes represent each part of the type tree from assembly to the class members. Each node is clickable and brings the tree focus to the clicked element.
Note (2): Look at the treemap, and image the nodes are you Namespace/TestFixture/TestCase hierachy. Failed test in red, successfull test in green. Just imagine, that would be the ultimate progress bar for MbUnit!
Note (3): Thanks to Jamie (NUnitAddin) for the idea sharing on this.

posted on Wednesday, June 16, 2004 12:52:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [2]

Jamie Cansdale, NUnitAddIn man, pointed out to me that I was cited (very briefly) on .Net rocks.  Here's the summary

John and Barry talk with us about Test-Driven Development, Unit Testing, and other aspects of Extreme Programming that are being used today. We had given lip service to unit testing in past shows, but John and Barry were able to explain the benefits as well as the how-to of test-driven development

http://www.franklins.net/dotnetrocks/, number 67, at 1:27 or so. This deserves a couple beers for me.

posted on Wednesday, June 16, 2004 12:36:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [7]
# Tuesday, June 15, 2004

This entry is a little tutorial on how to write your own Reflector Addin. For the past weeks, I have been playing a lot with those and Lutz Roeder was backing me up on MSN for quick questions, so it's my time to return him the favor and write a tutorial about it. So let's get started.A Reflector Addin is basically only a control that gets notified when the browser changes of active items. Of course, the tricky part is to "inject" your Addin into Reflector and to "clean" up when it is unloaded. Once, you're addin is injected, the control you provided takes care of the rest. I will not focus on writing controls here but rather on the procedure to load and unload packages.

Reflector API is rather big, so this tutorial is just the "immerged" part of the iceberg.

Quick Semantics

In Reflector, an Addin is called a Package (IPackage interface). Each package can add multiple Windows (IWindow), which are the control appearing on the right, can add CommandBar items (ICommandBarItem, ICommandBarButton) to the context menus, and attach listeners to the event trigerred when the AssemblyBrowser changes of ActiveItem (IAssemblyBrowser.ActiveItemChanged). The ActiveItem is the item selected in the reflection tree.

Building a simple package (hard way)

The IPackage interface is defined as follows:

public interface IPackage
{
   void Load(IServiceProvider serviceProvider);
   void Unload();
}

As may already know, the Reflector API is exposed through a set of interfaces. The actual implementations are obfuscated. The IServiceProvider instance is a "facade" against the implementations and can be used to retreive IAssemblyBrowser instance (and others). 

The Load method is where you will inject your Addin, while the Unload method is for cleaning. Implementing IPackage is rather boring and bug prone because you need to track all the window, menu item, etc.. that you inject to later remove them. Forgetting to remove a menu item is likely to happen often if you are in a rush. Moreover, using the serviceProvider is verbose in the sense that you access elements through a dictionary:

IAssemblyBrowser ab = (IAssemblyBrowser)serviceProvider.GetService(typeof(IAssemblyBrowser));

Let's implement a small package that displays a text box for assemblies. As mentionned above the steps are:

  • create your control and add it to the IWindowManager instance (retreive from serviceProvider),
  • add menu item and event handler,
  • set control to visible when menu is clicked,
  • add IAssemblyBrowser.ActiveItemChanged handler in your control
  • Don't forget to prepare cleaning up!

The control: this control display the name of the assembly, otherwise nothing.

public class AssemblyNameWindow : TextBox
{
    private IAssemblyBrowser assemblyBrowser=null;
    public AssemblyNameWindow()
    {
        this.Dock = DockStyle.Fill;
        this.Multiline=true; 
    }
    public IAssemblyBrowser AssemblyBrowser
    {
        get
        {
            return this.assemblyBrowser; 
        }
        set
        {
            if (this.assemblyBrowser!=null)
                this.assemblyBrowser.ActiveItemChanged -= new EventHandler(activeItemChanged);
            this.assemblyBrowser = value;
            if (this.assemblyBrowser!=null)
                this.assemblyBrowser.ActiveItemChanged += new EventHandler(activeItemChanged); 
        }
    }
    private void activeItemChanged(Object sender, EventArgs args)
    { 
        // testing if current item is an assembly
        IAssembly assembly = this.assemblyBrowser.ActiveItem as IAssembly;
        if (assembly!=null)
            this.Text=assembly.Name;
        else
            this.Text=null;
    }
}

And now the package, which creates a window, and some menu items:

public class AssemblyNamePackage : IPackage
{
    private IWindowManager windowManager=null;
    private ICommandBar assemblyMenu=null;
    private ICommandBarButton button=null;

    public void Load(IServiceProvider serviceProvider)
    {
        this.windowManager=(IWindowManager)serviceProvider.GetService(typeof(IWindowManager));
        IAssemblyBrowser assemblyBrowser = (IAssemblyBrowser)serviceProvider.GetService(typeof(IAssemblyBrowser ));

        // create window
        AssemblyNameWindow anw = new AssemblyNameWindow();
        anw.AssemblyBrowser=assemblyBrowser;
        // inject window into reflector
        this.windowManager.Windows.Add("AssemblyName",anw,"Assembly Name");

        // create menu item and attach handler
        ICommandBarManager cbm = (ICommandBarManager)serviceProvider.GetService(typeof(ICommandBarManager));
        // get assembly context menu
        this.assemblyMenu = cbm.CommandBars["Browser.Assembly"];
        // add button
        this.button = this.assemblyMenu.Items.AddButton("Assembly name",new EventHandler(button_Click));
    }

    public void Unload()
    {
        // remove window
        this.windowManager.Windows.Remove("AssemblyName");
        // remove button
        this.assemblyMenu.Items.Remove(this.button);
    }
    private void button_Click(Object sender, EventArgs args)
    {
        this.windowManager.Windows["AssemblyName"].Visible=true;
    }
}

That's it. You have built your first Reflector Addin:

Now this is a lot of code to get thigs up and moreover, it is quite errorprone, so let's us another easier method.

Building a simple package (easier way)

This solution uses a few classes that I have written in order to handle all the "load/unload" code. The new way is mainly based on custom attributes. [Update:]Firstly, you need to make your controls implement IServiceComponent where you can register to Reflector events:

public class AssemblyNameWindow : UserControl, IServiceComponent
{
    private ReflectorServices services = null;
    public ReflectorServices Services
    {
       get { return this.services;}
       set
       {
            if (this.services!=null)
            { ... (detach events)}
            this.services = value;
            if (this.services!=null)
            { ... (attach events)}
       }
    }
}

For example, the AssemblyPackage is rewritten as follows:

public class AssemblyPackage2 : BasePackage
{
    [ReflectorWindow(Caption="Assembly Name II")]
    [ReflectorCommandBar(CommandBarTarget.Assembly)]
    private AssemblyNameWindow assemblyName = new AssemblyNameWindow();
[Updated: no need to implement a property]
[Updated: no need to implement Load anymore]
}

This looks much nicer. BasePackage will self inspect and load all properties tag will ReflectorWindow attribute. For each one of those, it will add a menu entry to the corresponding menu using the information in ReflectorCommandBar attribute. A property can have multiple command bar targets and a package can have multiple windows. The example above gives as expected a new menu item:

Where's that package ?

The helper classes are not part of Reflector but you can get them from the MbUnit CVS (look in Samples/Reflector.Graph direcotry).

posted on Tuesday, June 15, 2004 10:12:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [6]
# Sunday, June 13, 2004

This release was sick so I removed from the server...

There is a new release of MbUnit available for download at tigris. The main change in this release is the support for separate AppDomain (which was really painfull to get).

New Features:

 

posted on Sunday, June 13, 2004 9:55:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [9]

In a previous post, I showed how you could use the IL graph to do static fault detection (at least for simple but recurrent faults). Since the code was using Reflector API, but it was more the job for FxCop rules. I started to port the code to FxCop.

As you can see on the screenshot below it is working, at least on my simple examples. In fact, it seems that FxCop decompiler is still a little buggy because it returns erroneous target offset for brtrue.s or brfalse.s IL instruction. Therefore, you will have to wait to play with this new toy :)

The figure below shows the FxCop rule applied to the test class showed in the previous post. As expected, ArgumentNotChecked fails, but the 2 others succeed the tests.

posted on Sunday, June 13, 2004 2:09:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [0]
# Saturday, June 12, 2004

New features:

Warning: This release has been compiled agains .NET v1.1 (because System.Drawing is buggy in v1.0), therefore you must create a Reflector.exe.config file in the directory of Reflector and add the following:







About bug reports:

Please, DO NOT post bugs in the blog. It is much more easy for me to monitor bugs through the http://mbunit.tigris.org issue tracking system. :)

Download Now on www.dotnetwiki.org

posted on Saturday, June 12, 2004 12:55:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [16]
# Friday, June 11, 2004

I've been reading lately about PreFast, a tool for detecting fault in applications based on static analyis. Sadly, PreFact is for C++, so I wondered: can we do the same in C# ?

The test fault:

So I started with simple fault: a method that uses nullable argument and that does not check for nullity -> the NullReferenceException fault (NRE). Here are three method that illustrate the problem:

public class MyClass
{
    public void ArgumentNotChecked(Object arg)
    {
        string s=arg.ToString(); // if arg is null, NRE
    }

    public void ArgumentChecked(Object arg)
    {
        if (arg==null)
            throw new ArgumentNullException("arg");
        string s=arg.ToString(); // ok, we know arg is not null
    }

    public void ArgumentGuarded(Object arg)
    {
        if (arg!=null)
        { 
            string s=arg.ToString(); // ok, we know arg is not null
        }
    }
}

Sketch of the solution

The solution (I found) to this problem can be understood as taking all the possible path in the IL graph, while keeping track of the current state of the argument: Null, NonNull or Uncertain. When starting, the argument is uncertain, it could be null or not. If a method instance of the argument is called 3 things can happen:

  1. the argument is Null: we have found a bug,
  2. the argument is Uncertain: we have a possible bug. If by design, you always check parameters, this is a bug,
  3. the argument is NonNull: that's ok :)

In terms of graph theory (and QuickGraph), we need to apply the EdgeDepthFirstSearch algorithm using a visitior (IEdgeColorizerVisitor) that the job  described above. More that words, let's explain this on schemas.

Checking the ArgumentNotChecked method

This methods is very simply. In IL, it looks like this:

ldarg.1
call Object.ToString()

The corresponding graph is the following (ColorCode: Orange=Uncertain, Red=Null, Green=NonNull)

The EdgeDepthFirstSearch has only one step: it explorezs the ldarg.1 -> ToString() edge. Since the ldarg.1 status when calling ToString is Uncertain, it should output a warning. So in Reflector I get:

Gotcha!

Checking the ArgumentChecked method

This method is already a bit more complicated since there is a branch. The IL is:

L_0000: ldarg.1 
L_0001: brtrue.s L_000e
L_0003: ldstr "arg"
L_0008: newobj instance void [mscorlib]System.ArgumentNullException::.ctor(string)
L_000d: throw 
L_000e: ldarg.1 
L_000f: callvirt instance string object::ToString()
L_0014: stloc.0 
L_0015: ret

and the IL graph looks like this:

The EdgeDepthFirstSearchAlgorithm starts by examining the ldargs.1 -> brtrue.s edge.

If we find a brtrue.s, we can easily check if it is applied to ldarg1. If yes, then we should update the argument state in the child edges. The "then" branch will be tagged Null, the "else" branch will be tagged NonNull  as depicted in the figure below: 

Now, when we encounter the ToString() call on the argument, it's state is set to NonNull, so that's ok. 

Now, in Reflector, the fault analysis on the method should not find any warnings:

It worked!

Conclusion

Fault analysis can be seen as a natural application of graph theory and QuickGraph is a good for doing this. Although Reflector is a great tool, I think fault detection needs to be implemented as FxCop rules.... so stay tuned for new adventures.

posted on Friday, June 11, 2004 5:25:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [5]
# Thursday, June 10, 2004

TypeGraph Addin:

This is the first preview of the next killer Reflector addin: Type Graph. The addin visualizes the type inherance thourgh a graph. If a namespace or a type is selected in the Reflector tree, the corresponding graph nodes are highlighted in green.

Why such an add-in ?

Besides outputing nice graphics, Type Graph has real software analysis potential (to my point of view). For example, it can be used to visualize Code Coverage: each day, your automated test automaton creates a big poster of the application, gray nodes are not covered, red have failed some tests and green are ok. You could even make a movie and play smart at the next TechEd :) And that's just the beginning...

Thanks to Jamie (NUnitAddin) suggested me the coverage application

Screenshot:

mscorlib graph with System.Reflection namespace highlighted

The algorithm are based on force directed layout which I found in Maxime Melchior final work (one of our student).

posted on Thursday, June 10, 2004 9:44:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [14]

This is a new add-in related to the ILGraph addin for Reflector. Basically I use the IL graph to extract all the path to cover the entire graph (and get full coverage). Each path then generates an empty Unit Test Case (using Refly) that the user will have to write. Better than words, the schema below is straightforward:

The addin can generate Unit Test cases for methods, types, namespaces or even full assemblies! Since it is based on CodeDom, it supports output to any .NET language.

The output

posted on Thursday, June 10, 2004 6:00:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [30]
# Wednesday, June 09, 2004

For those who want to play with a preliminary version of the PGF, here it goes:

Download PGF v1.0 (beta)

Update: the PGF is now part of TestFu.

posted on Wednesday, June 09, 2004 2:37:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [6]

Yesterday, I started reading Using Production Grammar for Software Testing (UPGS) for Emin Sirer and N. Bershad (found on ModelBasedTesting site), the authors describes a method for generating test automaton based on "inversed" grammars. Following their ideas, I have designed a little framework, the Production Grammar Framework for .NET (PGF).

Introduction

In this blog, I will present a framework for defining and executing production grammars. The production grammars described in UPGS are very similar to classic "LALR" grammars, in terms of understanding and semantics. The difference is that production grammars reversed the work of grammar, since they are used to produce output instead of analyse input.

Production grammars can do anything, ranging form generating data, piloting your application and generating test case, ... Since we are looking at testing applications, here are a few application that we will be interrested on:

  • manipulate the Implementation Under Test (IUT),
  • generate test case that manipulate the IUT. In this case, the grammar is a code generator,
  • if you want to test IUT against input data, generate a wide range of possible inputs,
  • etc...

Oracle problem

"Production grammars are well-suited for test generation not only because they can create diverse test cases effectively, but also because they can provide guidance on how the test cases they generate ought to behave. It is often hard to determine the correct system behavior for automatically generated test cases -- a fundamental difficulty known as the oracle problem. In the worst case, automated test generation may require reverse engineering and manual examination to determine the expected behavior of the system on the given test input."  The authors propose two ways of dealing with the oracle problem:

  • comparing difference with another implementation,
  • generating verification code along with the test code.

Since the PGF is based .NET code, you can theoretically add any verification code and if available, compare with any other implementation. This will be illustrated in the example

What is a Production Grammar:

"A production grammar is a collection of non-terminal rules to terminal rules that resembles a regular parsing grammar, but is used “in reverse.” That is, instead of parsing a sequence of tokens into higher level constructs, a production grammar generates a stream of tokens from a set of non-terminals that specify the overall structure of the stream".

Let's see that on a simple example. Consider the following comma separated list of names:

John,Paul,Marc

An valid EBNF grammar to parse this stream would be ([A-Z] designs capital letters, [a-z] lower case letters, + one or more occurence, * zero or more occurence):

name := [A-Z] [a-z]+
names := name (',' name)*

In the case of production grammar, we want to create that list of name. In C#, a possible implementation of the production grammar would be as follows:

public class NamesGrammar
{
   public string Name()
   {
        int count = nextRandom(); // gets a random int
        return getUpper() + getLowerString(count); // getUpper return upper case, getLowerString returns lower cased string
   }
   public string Names()
   {
        int count = nextRandom();
        string names = Name();
        for(int i=0;i!=count;++i)
           names += ',' + Name();
        return names;
   }
}

The example will generate a list name when the Names method is called, we have production :). This is just a dummy example, it is 100% hard-coded and very difficult to maintain. 

Production Grammar Framework

Before starting to work on an example (guess which), let us define important notations/interfaces of the PGF:

  • a rule (IRule) is an object that produce something. A rule can be terminal, it does not have any sub-rules, or non-terminal. Example of non-terminal rule is the + operator, terminal rule would be writing a string to a stream.
  • a production (IProduction) represents a production process (you can think of it as batch). It is passed on between the rules and can be stored to transmit the context or finish production,
  • Each time a rule wants to produce, it asks the production for a production token (IProductionToken). If the request is succesfull, the production goes on; otherwise, the production finishes. Using this mechanism, you can easily design production that will stop on a finite number of rule production.
  • a grammar (IGrammar) encapsultates a start rule and is used to launch the production process,

Those 5 interfaces form the kernel of the PGF, the rest of the framework is built on top and around them to help the user implement grammars.

The working example: a Stack

In order to crystilase the ideas presented here, we will consider the problem of testing a Stack (as in Test Driven Developement in .NET). A simple unbounded stack can be formalized by the following interface:

public interface IStack
{
    void Push(Object);
    Object Pop();
    Object Peek(); // equivalent to Pop() in TDD
    bool IsEmpty {get;}
}

The grammar

Before writing down the grammar in C#, it is useful to formalize it in a "pseudo" EBNF-like notation. In this example, there are 3 terminals: Push, Pop, Peek (we omit IsEmpty for now):

push := stack.Push(...)
pop := stack.Pop()
peek := stack.Peek()

 Since Pop and Peek can throw if the stack is empty, we create a special non-terminal rule  that guards production agains a specific exception type. This gives us the 2 following non-terminal rules:

gardedPop := gard( pop, InvalidOperationException )
gardedPeek := gard( peek, InvalidOperationException )

Of course, we need to be able to decided wheter we want to execute the guarded rule or the non guarded. Again, we define a new non-terminal rule that works as a "if-then-else". This gives us 1 more non-terminal rule:

nonEmpty := push | pop | peek
empty := push | gardedPop | guardedPeek
stackRule := if(isEmpty) { empty } else { nonEmpty }

The final step is to decide how many execution we want. Since we can break the production at the IProduction level, we go for infinite:

stackGrammar := stackRule*

The full grammar can be summarized as

push := stack.Push(...)
pop := stack.Pop()
peek := stack.Peek()
gardedPop := gard( pop, InvalidOperationException )
gardedPeek := gard( peek, InvalidOperationException )
nonEmpty := push | pop | peek
empty := push | gardedPop | guardedPeek
stackRule := if(isEmpty) { empty } else { nonEmpty }
stackGrammar := stackRule*

At this point, we have define a grammar for piloting the stack written in a "pseudo"-EBNF language. The simplicity of the notation let us concentrate on the problem and not on the tool to solve it. However, as it is, the grammar is pretty much useless, since it is missing verification code. We will see that the verification code can be easily added while writing the grammar in C#.

Rewriting the grammar in C# with PGF

The PGF contains a bunch of built-in non-terminal rules and helpers that will help you translate a grammar into functional C# code. In this example, we will encapsulate the grammar in a StackFixture class that contains a stack field:

public class StackFixture 
{
    private Stack stack = new Stack();
}

Step 1: Creating the terminal methods

Following the way we constructed the grammar, we begin by defining the terminals. The easiests solution is to define one method for each terminal and then wrap it into a IRule. For example, the push terminal is implemented as follows:

public class StackFixture 
{
    private Stack stack = new Stack();
    private IRule push;

    public void Push()
    {
        this.stack.Push(null);
    }

    public void CreateGrammar()
    {
        this.push = Rules.Method(new DefaultMethodDelegate(this.Push));
    }
}

We have added 3 things: a IRule field member to store the "push" terminal, a Push method that actually pushes something on the tested stack and a CreateRules method that will be used to creates the grammar. The Rules.Method is a static helper class that encapsulate the call to a member method into a rule. The same operation can be repeated for pop and peek.

Step 2: Adding code for the Oracle problem

In order to be able to verify assertion on the fixture, we need to solve the Oracle problem. For this specifiy case, we can do the Comparaison testing approach where we execute the operations on another implementation and verify that outputs are different (NC, not NSC). 

public class StackFixture 
{
    private ArrayList list = new ArrayList();
    private Stack stack = new Stack();
    private IRule push;

    public void Push()
    {
        this.stack.Push(null);
        this.list.Add(null);
        Assert.AreEqual(list.Count==0, stack.IsEmpty);
        Assert.AreEqual(list[list.Count-1], stack.Peek());
    }
    ...
}

Step 3: Creating the non-terminal rules

From this step, the PGF provides the "classic" non-terminal rules and helper classes to create them (Rules) that makes the rest of the implementation straightforward:

public class StackFixture
{
    ...
    IRule push, pop, peek;
    IRule guardedPop, guardedPeek;
    IRule empty, nonEmpty;
    IRule stackRule;
    IRule stackGrammar;
    ...

    // returns true if stack is empty
    public bool IsEmpty()
    {
         return this.stack.IsEmpty;
    }

    public void CreateGrammar()
    {
        push = Rules.Method(new DefaultMethodDelegate(Push));
        pop = Rules.Method(new DefaultMethodDelegate(Pop));
        peek = Rules.Method(new DefaultMethodDelegate(Peek));

        guardedPop = Rules.Guard( pop, typeof(InvalidOperationException) ); // gard( pop, InvalidOperationException)
        guardedPeek = Rules.Guard( peek, typeof(InvalidOperationException) );// gard( pop, InvalidOperationException)

        nonEmpty = Rules.Alt( push, pop, peek); // push | pop | peek
        empty = Rules.Alt( push, guardedPop, guardedPeek);
 
        stackRule = Rules.If(new ConditionDelegate(IsEmpty), empty, nonEmpty); // if(IsEmtpy) empty else nonEmpty

        stackGrammar = Rules.Kleene(stackRule); // stackRule*
    }
}

Step 4: Creating a grammar and executing a production

In order to simplify our lives, we make StackFixture inherit from the Grammar class (which implement IGrammar) provided by the PGF. This class takes care of setting up the production factory and launching productions. Of course, this part of the framework can be easily customized and extended to plugin into your existing test automaton (MbUnit, NUnit, csUnit, ...).

public class StackFixture : Grammar
{
    public StackFixture()
    { 
        this.CreateRules();
    }
    public void CreateRules()
    {
        ...
        // required for Grammar
        this.StartRule = this.stackGrammar;
    }
}

A production can then be executed by launching the Produce method:

StackFixture fixture =new StackFixture();
fixture.Produce();

Conlusion

Production Grammar Framework is an effective tool to create and implement production grammar. Production grammar are used to model the complex behavior of application and automate the creation of complex test case. In fact, you could easily reuse your existing Unit Test cases with a production grammar! Of course, PGF will be part of MbUnit in the next release. Moreover, since the "hard" work is done by the machine, you can expect to get much better code coverage using Production Grammar (or MBT) than with classical unit tesing.

Future directions and the quizz

This post is already long so I will stop here. However, you can start to think about other interresting use of this technique:

  • database testing,
  • data generation for user input testing,
  • unit test case generation using CodeDom,
  • etc...

The quizz question: How does this relates to model based testing ?

posted on Wednesday, June 09, 2004 11:54:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [15]
# Sunday, June 06, 2004

With the growing need for a decent .NET wrapper of LAPACK and since there exists no free wrapper around it (commercial version here), I have decided to take a try at this adventure (working with managed C++ is always an adventure). To help me in the task, Lapack++ a C++ wrapper around LAPACK: simple and very well done, should speed up things... (to be continued)

posted on Sunday, June 06, 2004 10:42:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [7]

Reflector.Graph is a Reflector 4.0.5.0 Add-in. (it is my new toy project). Actually, it contains 3 graph-based package for Reflector (more will come )

Don't forget to drop a line to tell me about the tool.

Go to Reflector.Graph v1.1

Hint: Click and move to zoom in/out of the graphs
Hint2: Make sure you download Reflector 4.0.5.0
Hint3: Load Reflector.Graph.dll assembly in the Addin dialog.

posted on Sunday, June 06, 2004 10:08:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [21]

After a number of requests, QuickGraph assemblies have been merged. Here is the new layout:

 

posted on Sunday, June 06, 2004 9:27:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [5]
# Friday, June 04, 2004

This is a new experimental Reflector Add-In that will be part of the upcoming Reflector.Graph addin (see Assembly Grapher and Il Grapher). The objective of this addin is to rank methods using a method similar to google.

The graph

Consider a graph where the vertices are the methods of the classes, and each edge represents the fact that the source method calls the target method. For example, take the following dummy class:

public class Test
{
   public void Hello()
   {...}
   public void HelloWorld()
   {
       this.Hello();
   }
}

The method graph from this class will contain

  • 2 vertices: Test.Hello and Test.HelloWorld
  • 1 edge: Test.HelloWorld -> Test.Hello.

PageRank, The Algorithm

PageRank, which is the algorithm that Google uses to rank pages, is a well known and easy to implement algorithm (it is implemented in QuickGraph). The idea behing is very simple and elegant: important pages are reference by other important pages. So basically, PageRank algorithms can help you order the vertices of your graph by order of importance.

PageRank and MethodRank

Contrary to Google, where the vertices are pages and edges are the hyperlinks, the vertices are methods. However, the PageRank idea translates naturally to the method graph: important methods are called by other important methods. Why would we need to know which are the important vertices ? Here are my intuition (not tested) about this:

  • if a important method contains a bug, it is likely that a lot of test cases will fail, (this a BIG assumption)
  • using the previous point, it is clear that important methods are a good target for mutation testing,
  • if you have to start testing, it can give you a "test writting" order,
  • it's fun

The AddIn

for the pleasure of the eyes

posted on Friday, June 04, 2004 8:55:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [10]

This post is just a small remainder of the meta-physical questions I'm asking myself, specially because testing is basically trying to solve an unsolvable problem (Halting theorem)

  1. What is a bug ? (I like this one)
  2. Are there family of bugs ? (see Mutation Testing) Can we detect them statically (i.e. using Metadata only)
  3. Is there a way to get an a-priori estimation on the location of the bugs ?
  4. Can we apply Risk Managment technology to source code ? Automatically ?
  5. How can you effectively test a database ? By effectively I mean: not restricted to transactions, not database reconstruction for each test,
  6. How can you effectively test a GUI ? By effectively I mean, without having to actually record and program user interfaction,
  7. Should you test the code that does the testing ? If yes, should you test the code that tests code that does the testing... and so on,
  8. You are facing an untested application, you don't have the time to test all the code. Where should you start ?

 

posted on Friday, June 04, 2004 2:24:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [2]

I have attended yesterday to a presentation about SharePoint by Patrick Tissegem and Inge De Neef (?could not find her blog?). Inge, sorry for the question :). Patrick was presenting through a VPN and doing live stuff on this SharePoint site (Inge also did live manipuation). It did not crash, amazing. This was my first contact with SharePoint and it looked like a terrific product for creating intranet, but it also appeared to me that it could quickly become a mess if you start tweaking it!

I also met a lot of fellow belgian bloggers. Since I'm not good with names, I must have forgotten many but here are the people I saw and spoke to: Jan Tiellens, Tom Mertens (MSDN content manager I think), Thomas Delrue (who will be doing an internship at Microsft as SDE/T this summer), and I forgot the other... Make yourself heard :)

There was also a surprise from the MSDN team before the presentation where Tom said:

"We have two .Net gurus in the room[...] I would like you to give a warm applause to Jonathan de Halleux and Thomas Delrue[...]".

I must say I was not easy with that, but he then said:

"Microsoft has a gift for both of them[...] bottles of champagne[...]"

Then I was much more relax :) :) :)

posted on Friday, June 04, 2004 2:05:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [0]

That's another graph-based Reflector Add-In. It creates and displays the dependency graph between the assemblies that are loaded in Reflector.

ps: Thanks to Lutz who wrote the AssemblyGrapher application to make it a Reflector add-in.

posted on Friday, June 04, 2004 7:28:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [15]
# Wednesday, June 02, 2004

I have received an interresting post from Kent Boogaart on the CodeProject article that pointed out a major limitation of Composite Unit Testing:

I'm a little unsure how this tests all an interface's members. Take ICollection, for example. Your CollectionTest implementation would receive an instance of ICollection manufactured via an appropriate factory.

Say you wanted to test that ICollection.Count was returning the correct value. How would you test this without knowledge of the factory method called?

Thanks,
Kent

There he has a point. You cannot test ICollection.Count using Composite Unit Testing as it is now. You could test it through the IList test interface, but that's not the point.

A way of overcoming the limitation would be the ability for the user to give a "gold" instance along with the test instance. The gold instance would be mock of the interface as it "should" be and could be used to do the ICollection.Count test. For each factory method, the framework would look for it's mock counter part, if found, it is feeded to the test method. Let's illustrate that on ICollection:

The gold instance class (mock):

// the mock
public class CollectionMock : ICollection
{
    private int count=0;
    ...
    public int Count
    { 
        get{ return count;}
        set{this.count=value;}
    }
    ...
}
The tested instance factory
public class ArrayListFactory
{
    // provider for empty arraylist
    [Factory] // new attribute, tells the framework it is a factory method
    public ArrayList Empty
    {
        get
        {
            return new ArrayList();
        }
    }
    [MockFactory] // tells the framework it is a mock method
    public MockCollection EmptyMock // name is important must be "Method"+Mock
    {
        get
        {
            MockCollection col = new MockCollection();
            return col;
        }
    }
    // 1 provider + mock
    [Factory]
    public ArrayList One
    {
        get
        {
            return Empty.Add(null);
        }
    }
    [MockFactory]
    public MockCollection OneMock // name is important must be "Method"+Mock
    {
        get
        {
            MockCollection col = new MockCollection(); 
            col.Count=1;
            return col;
        }
    }
}
The test fixture:
[TypeFixtue(typeof(ICollection)]
[ProviderFactory(typeof(ArrayListFactory),typeof(ICollection))] public class CollectionTest { [Test] public void CountTest(ICollection tested, ICollection mock) { Assert.AreEqual(mock.Count,tested.Count,"ICollection.Count"); } }
The framework would detect if a mock is available and automatically feed it to the method. Any comments ?
posted on Wednesday, June 02, 2004 8:57:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [4]

I have been facing a silly bug related to AppDomain, so stupid that it is worth mentionned in a blog.

The symptoms

Load an assembly in a separate AppDomain, create an instance of a type. Things work correctly. Then unload the domain, and recreate the type. Somehow, strange bugs (Custom Attributes not detected, the debugger messed up) arise.

How-to recreate it:

Assume that we have the Assembly MbUnit.Core.dll, MbUnit.Framework.dll (that references Core) and Test.dll that references both. Core defines an abstract Custom Attribute TestPatternFixtureAttribute,

public abstract TestFixturePatternAttribute :Attribute{...}
that is inherited in Framework as TestFixtureAttribute
public class TestFixtureAttribute : TestFixturePatternAttribute{...}
and TestFixtureAttribute is used in Test to tag fixture classes
[TestFixture]
public class MyTest{...}
At last, Core contains RemoteTestTree a serializable class that takes care of loading assemblies, extracting fixtures (class taged with TestPatternFixtureAttribute) and launching tests.

  1. Create a separate AppDomain using AppDomain.CreateDomain,
  2. Create an instance of RemoteTestTree using AppDomain.CreateInstanceFromAndUnWrap
  3. Load Test.dll in the domain and find fixtures. At this point, everything works and we found the fixtures.
  4. Unload the separate AppDomain using AppDomain.UnLoad,
  5. Re-Create an instance of RemoteTestTree (at this point, weird things happen because the debugger seems to tell: RemoteTestTree is not initialized)
  6. Load Test.dll in the domain and find fixtures. This time, no fixture is found because the framework seems to tell that TestFixtureAttribute does not inherit from TestPatternAttribute.

The fix

In point 2,5, I have been using AppDomain.CreateInstanceFromAndUnWrap, this is the error. I should have been using AppDomain.CreateInstanceAndUnWrap. The difference is subtle, 4 letters.

The explanation

I'm still unsure of the explanation.... so I'm waiting for someone to come up with a relevant explanation :)

posted on Wednesday, June 02, 2004 6:37:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [9]