# Thursday, June 24, 2004

Little intro

Reflector is a (great) decompiler. The most obvious use of it is to load an assembly and recreate source code out of the IL it contains. The assembly itself was built from source code, so we have a mini "cycle of life". In a perfect world, the decompiled code and the orginal source code would be exactly the same (at least in the same language). So we can represent this cycle by the figure below.

Now,  let's introduce a new actor in this ecosystem. We want the decompiler to producec CodeDom source code that can be used to generate the original source code as shown in the figure below: we want to produce a CodeDom template that will generate that type.

Sample example

Let's illustrate all this with an example. Consider the following User class:

public class User
{
    private string name = "Marc";

    public String Name
    {
        get
        {
            return this.name;
        }
        set
        {
            this.name=value;
        }
    } 
    public override string ToString()
    {
        return this.name;
    }
}

We compile User and now, we want Reflector to generate a class that uses CodeDom (indeed Refly) to generate back the User class. This is the tricky part. I need to use Refly to generate code that uses Refly, so that would be building a code generator generator. The template for the User class will look like this:

public class UserTemplate 
{
    private Refly.CodeDom.ClassDeclaration _type;
    private Refly.CodeDom.FieldDeclaration _nameField;
    private Refly.CodeDom.PropertyDeclaration _nameProperty;
    private Refly.CodeDom.MethodDeclaration _toStringMethod;

    public UserTemplate(Refly.CodeDom.NamespaceDeclaration ns) 
    {
        // add User class to generator
        this._type = ns.AddClass("User");
        this.Declare();
    }

    public virtual void Declare() 
    {
        this.DeclareFields();
        this.DeclareMethods();
        this.DeclareProperties();
        this.DeclareEvents();
        this.DeclareInnerClasses();
    }

    public virtual void DeclareFields() 
    {
        // adding name field
        this._nameField = this._type.AddField(typeof(String), "name");
    }

    public virtual void DeclareMethods() 
    {
        this._toStringMethod = this._type.AddMethod("ToString");
        this.PopulateToStringMethod(this._toStringMethod);
    }

    public virtual void DeclareProperties() 
    {
        this._nameProperty = this._type.AddProperty(typeof(String), "Name");
        this.PopulateNameProperty(this._nameProperty);
    }
    ...
}

(The above template is still missing a big part: the generation of the statements and the expression) Now that we have a Refly ClassDeclaration, we can feed it to the CodeGenerator class:

NamespaceDeclaration ns = new NamespaceDeclaration("RoundRobin");
UserTemplate template = new UserTemplate(ns);
CodeGenerator gen =new CodeGenerator();
gen.GenerateCode("./Output",ns);

And the result of the execution of this program will be this piece of code:

namespace RoundRobin 
{
    using System;
    public class User 
    {
        private string _name;
        public virtual string Name 
        {
        }
        public virtual void ToString() 
        {
        }
    }
}

Not so bad for a start, we have gotten our User class back but there are still some big "holes" in it....

posted on Thursday, June 24, 2004 8:40:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [0]

Following the idea of displaying a class in a PropertyGrid, we can also take a look at how it serializes using the XmlSerializer (see if it serializes at all!). The task looks trivial at first sight:

  1. create an instance of the object,
  2. create a XmlSerializer that can serialize it,
  3. write the object to a XmlRichTextWriter to get colors :)

Of course, there's a gotcha here. You will likely get empty xml files all of the time because usually, the default constructor creates "empty" object. So what we want is to dress the object before serializing them. This is done in a variety of ways depending on the property type:

  • the property/field is a string, int, etc... you can set easily a value to it,
  • the property/field is tagged with XmlArrayAttribute,
    • the property/field is an array: get all the XmlArrayItem attribute, create an array and fill it with instance of those items,
    • the property/field is a collection: get the "Add(Object)" method, add instances of the items.

Of course, this is rather simplistic and may fail on forgotten situations.

Screenshot: ReportAssembly contains collection properties (namespaces, fixtures),  which would appear empty if not "dressed".

posted on Thursday, June 24, 2004 7:23:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [2]
# Wednesday, June 23, 2004

In this blog, I'll show how you can a implement a control to colorizes XML in a simple and effective way. I needed to display XML in a control, however, displaying it in "black and white" made it very depressing.  I have 3 objectives: simplicity, simplicity and simplicity.You can think of this as a traditional Microsoft interview enigma. So let's enumerate what we need and what we want (and what we want to avoid). 

What we want: colors!
What we don't want: manipulate the XML Dom, in other words: let's be lazy.
What we have: it's time to dig into the FCL classes and see what we could use 

  • RichTextBox: this control seems to be the perfect match for our "coloring" purposes,
  • XmlTextWriter: this class is a forward only, non-cached Xml writer. This class provides a rich set of method that output elements, attributes, etc...

Implementation

At this point, you must have an idea of the way we are going to acheive colors... the tools are here, we just need to inject the color at the right spot (like in a toothpaste, the color lines come right at the end). In fact, we have 3 tasks to acheive:

1) RichTextBoxWriter:

A TextWriter-derived class that writes to a RichTextBox. The writer keeps a current Color and Font that is apply to each chunk of string appended. This task is pretty straightforward, basically you need to overload 3 Write methods.

public class RichTextBoxWriter : TextWriter
{
    private RichTextBox textBox;
    private Color currentColor=Color.Black;
    private Font currentFont = new Font("Tahoma",8.25f);

    ...

    public override void Write(char value)
    {
        this.TextBox.SelectedText = value.ToString(this.FormatProvider);
    }
    public override void Write(char[] buffer)
    {
        if (buffer!=null)
        {
            this.TextBox.SelectedText = new String(buffer).ToString(this.FormatProvider);
            // if buffer = '\n\r', font and color is reseted
            this.TextBox.SelectionColor=this.CurrentColor;
            this.TextBox.SelectionFont=this.CurrentFont;
        }
    }
    public override void Write(string value)
    {
        this.TextBox.SelectedText = value.ToString(this.FormatProvider);
    }
}

Wow, that was short. Now to point 2.

2) XmlRichTextWriter:

This class, which inherits from XmlTextWriter, injects color on some "startegic" methods. In order to store the colors, we create XmlRichTextBox, which inherits from RichTextBox and contains a rich set of customizable colors and fonts for the Xml rendering. This class will be used by the XmlRichTextBoxWriter:

public class XmlRichTextBoxWriter : XmlTextWriter
{
    private XmlRichTextBox theme; // contains all the colors and fonts

    public XmlRichTextBoxWriter(XmlRichTextBox tb)
    :base(tb.Writer)
    {
        this.theme=tb;
    }

    public override void WriteStartDocument()
    {
        // setting color of the selected text
        this.theme.Writer.CurrentColor = this.theme.StartColor;
        this.theme.Writer.CurrentFont = this.theme.StartFont;
        // output data
        base.WriteStartDocument();
    }
    ...
}

4) XmlRichTextBox:

Now, that we have our writer, we just need to wire them up in the XmlRichTextBox and we are done.

And finally, the result: on the left, a XmlRichTextBox, on the left, how it looks like in the PropertyGrid. We have colooors!

Download it now....

posted on Wednesday, June 23, 2004 9:08:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [6]

Joel Pobar speaks about the new features of Reflection.

ReflectionOnly seems to be very interresting...

posted on Wednesday, June 23, 2004 9:14:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [1]
# Tuesday, June 22, 2004

Ever wondered how you classes you look like in a PropertyGrid control, now it is possible directly from Reflector with my upcoming Reflector Addin:

Ps: As always, a big thanks to Jamie Cansdale (NUnitAddIn) for his help on this one.

posted on Tuesday, June 22, 2004 4:19:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [5]

XsdTidy is a refactoring tool to overcomes some silly limitations of the exceptional Xsd.exe  tool provided with the .NET framework. More specifically, XsdTidy addresses the following problems:

  • Name normalization: if your XSD schema is using lower case names or more generally non ".NET" normalized names, you will end up with types that will make the FxCop spit out hundreds of infractions,
  • Fields not properties: xsd.exe creates fields and does not "hide" them in properties which is bad OO design.
  • Fixed Array Sizes: xsd.exe handles multiple elements by creating an array. There is no problem when you are loading the data, but unfortunately this is not convenient if you want to populate a document since arrays do not support Add or Remove. XsdTidy provides strongly-typed collection that support Add, Remove, etc...
  • Default Constructor: Xsd.exe does not care about providing a default constructor that initializes the fields with the proper values. If the object structure is getting big, it becomes very difficult to properly initializes fields,
  • Serializable: Xsd.exe does not tag classes with serializable

Note that XsdTidy uses Refly for building the source code and is also maintained by Marcus Mac Innes.

What does XsdTidy fix ?

Name conversion

The .NET standards define specific naming convention for all types of data: arguments should be camel case, function names capitalized, etc... This is really helpful to keep the framework consistent. Tools like FxCop help us stay on the "normalized" side.

This problem is tackled the dumb way: given a dictionary of "common" words, the class NameConformer tries to split a name in separate words, after that it renders it to the needed convention. Of couse, this feature can be disabled.

FixedArraySize and "Multi" Strongly-Typed Collections

Arrays are replaced by inner strongly-type collections which are much more flexible to use. Moreover, array fields are created by default using their default constructor. This is to economize you the hassle of creating a collection before using it. If an array can support multiple object type, the generate collection will be "multi" strongly typed:

public class TestClass
{
    [XmlArray("values")]
    [XmlArrayItem("car",typeof(Car));
    [XmlArrayItem("car",typeof(Bike));
    public Object[] values;
}

becomes

public class TestClass
{
    private ValueCollection values = new ValueCollection();

    [XmlArray("values")]
    [XmlArrayItem("car",typeof(Car));
    [XmlArrayItem("car",typeof(Bike));
    public ValueCollection Values
    {
        get { return this.values;}
    }

    public class ValueCollection : CollectionBase
    {
        public void AddCar(Car car)
        {
            this.List.Add(car);
        }
        public void AddBike(Bike bike)
        {
            this.List.Add(bike);
        }
        ... // the implementation of the collection
    }
}

Properties

Fields are hidden in properties, which is more convenient to use. Moreover, collection fields do not have set property according to FxCop rule.

public class testclass
{
    [XmlAttribute("values")]
    public String values;
}

becomes:

public class TestClass
{
    private String values;

    [XmlAttribute("values")]
    public String Values
    {
        get
        {
            return this.values;
        }
        set
        {
            this.values = value;
        }
    }
}

Serializable

The output classes are tagged with the Serializable attribute to make them usable using Remoting.

XsdTidy history

I have first started to build XsdTidy using System.Reflection.Emit. It was a titanic job and very error prone. The difficulty of using Emit pushed me to use CodeDom which was also heavy to use. So finally, Refly was designed and XsdTidy became much easier to impement.

Download:

Download XsdTidy and Refly at http://blog.dotnetwiki.org/downloads/Refly.zip

Screenshot

posted on Tuesday, June 22, 2004 9:33:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [6]
# Monday, June 21, 2004

Reflector.Graph was becoming messy because of external dependencies so I decided to remove some parts of it (TypeGraph, TreeMap) in order to keep funcitonal.

Reflector version: tested for 4.0.6.0 and 4.0.7.0

Installation procedure:

  • Download the Reflector.Graph Binaries  and unzip it in the Reflector directory (it may be unzipped in a Release directory, copy the file aside to Reflector)
  • In Reflector, Tools -> Addin... -> Add -> Reflector.Graph.dll
  • That's it!

In this release, you do not need to create a Reflector.exe.config.

What's in Reflector.Graph:

  • MethodRank (Assembly),
  • TypeRank (Assembly),
  • Unit Test Generator (Type),
  • Mock Geneator (Type),
  • IL Graph (Method),
  • AssemblyGraph (Tools)

What's not in Reflector.Graph:

  • TypeGraph will be release separately because it needs to be compiled agains .Net v1.1
  • TypeTreeMap will be released separately because it has a dependency on the Microsoft TreeMap

Bugs and suggestions:

 

posted on Monday, June 21, 2004 8:56:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [2]
# Sunday, June 20, 2004

The Database Populator Framework  follows the same idea of DbMonster (or at least, was partially inspired from it). The purpose of the framework is to provide a flexible (and smart) set of random data generators to populate your database.

Why ?

If you plan to do database testing, the solution are not much: use transaction, create/kill database after each unit test, clean up after each test or use the new technique using Enterprise Services.

The objective of DPF is not to test the database but provide "food" for your unit test. By generating data randomly (satisfying the constraint), you do not need to clean up and you will avoid test clashing on each other. Moreover, you can also use the DPF to test your database under load.

Random but no dumb

In order to generate data for testing database, you need to take into accounts the constraints on the columns and between the tables. By providing a DataSet representing you database, the framework analyses it and create a set of generators. The data generated satisfies all the constraint. Of course, each row generator can be customized for your own needs, but in general, the information contained in the dataset is enough.

The ideas

The idea are rather simple:

  • Each DataColumn has its corresponding IDataGenerator instance,
  • Each DataTable has its corresponding ITablePopulator instance,
  • Each UniqueConstraint has its corresponding IUniqueValidator instance which takes care of ensuring the all unique constraint are validated,
  • Each ForeignKeyConstraint has its corresponding IForeignKeyConstraint instance which will fetch valid foreign key values

The generation of a new row is made inside a ITablePopulator as follows:

  1. Create a row,
  2. Call each IDataGenerator and fill the row,
  3. For each foreign key, get relevant values,
  4. Verify unique constraints, if violated go to 1,
  5. return row

The first bits

The first bits of the framework is located in the MbUnit.Framework.Data namespace. Althout the framework still needs work, the first example is running and is quite promising.

I have built a simple database containing User - Order - Product - OrderProduct:

this.dataSet=new DataSet();

DataTable users=dataSet.Tables.Add("Users");
DataColumn userID = users.Columns.Add("UserID",typeof(int));
DataColumn userName=Users.Columns.Add("UserName",typeof(string));
DataColumn userName.AllowDBNull=false;

DataTable orders=dataSet.Tables.Add("Orders");
DataColumn orderID=orders.Columns.Add("OrderID",typeof(int));
DataColumn orderDate = orders.Columns.Add("OrderDate",typeof(DateTime));
DataColumn oUserID = orders.Columns.Add("UserID",typeof(int));

DataTable products=dataSet.Tables.Add("Products");
DataColumn productID=products.Columns.Add("ProductID",typeof(int));
DataColumn productName = products.Columns.Add("ProductName",typeof(string));
DataColumn productPrice = products.Columns.Add("ProductPrice",typeof(decimal));

DataTable orderProducts=dataSet.Tables.Add("OrderProducts");
DataColumn opOrderID=orderProducts.Columns.Add("OrderID",typeof(int));
DataColumn opProductID=orderProducts.Columns.Add("ProductID",typeof(int));
DataColumn quantity=orderProducts.Columns.Add("Quantity",typeof(int));
// pks
users.Constraints.Add("PK_Users",userID,true);
orders.Constraints.Add("PK_Orders",orderID,true);
products.Constraints.Add("PK_Products",productID,true);
orderProducts.Constraints.Add("PK_OrderProducts",
new DataColumn[]{ opOrderID, opProductID}
,true);
// fks
orders.Constraints.Add("FK_Orders_Users",userID,oUserID);
orderProducts.Constraints.Add("FK_OrderProducts_Orders",orderID,opOrderID);
orderProducts.Constraints.Add("FK_OrderProducts_Products",productID,opProductID);

This database is easily populated using the DPF:

// creating populator
IDatabasePopulator pop=new DatabasePopulator();
// anaylising internal structure
this.pop.Populate(this.db.DataSet);

After that, you can customize the behavior of every data generator. Once this is done, you are ready to "feed" your database:

//getting the users table populator
ITablePopulator userPop = pop.Tables[users];
// adding new row
users.Rows.Add( userPop.Generate() );
posted on Sunday, June 20, 2004 11:03:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [8]

Refly is the library that I have developped to make CodeDom usable. (See my CodeProject article here, or the classic Hello World example). Note that Refly comes with XsdTidy, an application that "pretify" the output of the Xml.exe tool.

Download Now

 

posted on Sunday, June 20, 2004 10:17:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [3]

If you are doing database testing, you should read this great post by Roy:

Simplified Database Unit testing using Enterprise Services

posted on Sunday, June 20, 2004 8:31:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [1]