Since it’s been almost a year that we’ve released Pex, we have started a little survey on our forums. We are looking for feedback on how you use Pex and how you would like to use it. If you are interested, please voice yourself  on our mini survey at

http://social.msdn.microsoft.com/Forums/en/pex/thread/27c703ea-3928-41ef-b767-638c39966b99

The questions are:

1. Are you using Pex already? A) From Visual Studio, B) the command-line, C) other, D) no
2. Which version of Visual Studio are you using? A) Professional, B) Team System, C) other
3. How many people are in your team?
4. Do you write A) Parameterized Unit Tests (PUTs), or B) only use the Wizard-generated PUT stubs?

Any other comments are appreciated as well, about the license terms, or required Visual Studio versions…
Thanks,  The Pex Team

While on parental leave, I could sneak a couple hours to read a paper on Frontier Search by Korf and al. Frontier search is a kind of best-first search algorithm that reduces the needs for memory. A very interesting and relaxing read… I add an implementation to QuickGraph that follows the idea.

The new release of Code Contracts is out and brings a very cool feature: Xml comment generation. This means that you do not have to worry about keeping the comments in sync with the code, the compiler takes care of this.

Contracts to Xml Comments in action

Unless you invest a lot of work in them, Xml comments are most often worthless. For example, in QuickGraph, the documentation of the Edge constructor is really… well… useless (oops my fault):

However, the body of the constructor contains Contracts that state the pre-conditions and post-conditions of the constructor: source and target should not be null, etc… With the new xml comment generation, these contracts will be added to the xml documentation and ultimately will show up in the compiled documentation.

First, let’s turn xml comment generation on in the property pages:

Once the project is rebuilt, we can take a look at the generated xml documentation file. The ‘requires’ and ‘ensures’ elements are now under the Edge constructor element:

Finally, we run the documentation file through Sandcastle*** to get the final result (make sure you update the Sandcastle stylesheets that come with Contracts. Read section 8 of the documentation):

Voila! With this feature, you have a documentation that never goes out of date.

I’ve just made a release of QuickGraph 3.3 on CodePlex. Along with the usual bug fixes, this version brings a set of new graph data structures based on delegates. Instead of taking care of the storage, these graphs simply ‘query’ delegates for vertices and out-edges. You can use them to bridge any existing graph data structure in your code with QuickGraph – with almost no performance/memory penalty. (Delegate graphs can also be used to deal with graphs that won’t hold in memory but that’s another story).

Let’s see this with an example. Suppose, we have a simple graph representation using a jagged array, int[][] graph, where each edge is defined as (i, graph[i][j]) :

In order to use this graph with QuickGraph, we ‘wrap’ it up into a delegate graph. There are a number of extension methods in GraphExtensions that take care of the common cases. Delegate graphs rely on 2 delegates: one that returns the vertices, the other one out-edges:

At this point, we can apply any of the algorithms that QuickGraph provides. For example, computing a topological sort of the vertices:

The new delegate graphs are ready to be downloaded at http://quickgraph.codeplex.com/Release/ProjectReleases.aspx .

Happy programming!

We just released a new version of Pex, v0.15.40714.1. This version does not contain any new features but… it brings better memory management and bug fixes for issues that were reported through our MSDN forums. Happy testing!

It contains a couple breaking changes too:

• Microsoft.Pex.Stubs.dll does not exist anymore and has been integrated into Microsoft.Pex.Framework.dll
• PexExplorableFromFactoriesAttribute renamed to PexExplorableFromFactoriesFromAssemblyAttribute

We have just released a new version of Pex, v0.14.040610.2. This version brings a range of Bug Fixes that were reported on the MSDN forums and a couple new features: Fix Suggestions for Invariant Methods and Natural properties  for Stubs.

Fix Suggestions for Invariant Methods

When you use Code Contracts, then Pex can now help you to add missing invariants, just as Pex already inferred missing preconditions in the past. Take for example the ArrayList class that comes with the samples of our recently updated tutorial slide deck. This class has the following fields:

When you add an empty invariant method,

and enable Runtime Checking of contracts, then Pex will create instances of this class using reflection while making sure that the invariants holds (we announced this feature in the last release). However, with an empty invariant method, Pex will be able to create illegal states, e.g. where the _items array is null, which may cause a NullReferenceException, and other issues. To get the following test cases, we let Pex explore the Add method of our ArrayList class:

When you look at the details of the failing test cases, then you will see that Pex now offers the option to add invariants directly into your [ContractInvariantMethod]:

Natural Properties for Stubs

The logic to handle properties with setters and getters generated by Stubs has been slightly improved so that it is possible to force properties to have as if they had a backing field.

When a getter or a setter of a property is invoked, Stubs checks whether the getter delegate or the setter delegates have been set. If not, i.e. no custom user behavior, it queries the fallback behavior for an initial value of the property. If the fallback behavior successfully returns a value, Stubs generates a closure and binds the getter and setter.

For example, here’s a simple IFoo interface with a Bar property:

One could write the following parameterized unit test:

Since we expect the Bar property to behave as it was wrapping a field, we don’t expect the assertion to be raised. When we execute this test with Pex, and the Pex fallback behavior, Pex will choose a value for ‘Bar’, and this value will be ‘stored’ the property. Therefore, we get as expected 2 test cases:

Non default constructor support for Stubs

In previous versions, Stubs could not generate stubs for classes with no default constructor. This is no longer the case, Stubs will generate one public constructor for each public or protected base constructor.

Bug Fixes

• Various code generation tweaks to make StyleCop happier
• Visual Studio addin crashes when loading in VS2010
• Visual Studio addin does not find attributes when they are globally qualified
• Added missing substitutions for System.Threading.Interlocked members
• Stubs automatically imports missing references
• When an object has an invariant method (from Code Contracts), Pex would emit assertions using the private fields
• Editorial issues to patterns paper
• Contracts no longer kill the process when the runtime rewriter was not executed (.net 4.0)
• Fixed bug in generic type guesser
• Fixed samples for the latest version of Stubs

As always, don’t hesitate to send us your feedback, good or bad, through our MSDN forums.

Inversion of Control (IoC) is a very important practice to make code testable. It usually relies on defining interfaces for each external dependency. Various frameworks and techniques, i.e. Dependency Injection (DI), Service Locator, exist to bind the interface implementations to the components. In this blog post, we will *not* focus on this aspects of IoC but rather on it’s main building block: interfaces. This article describes how Contracts for interfaces improves IoC… regardless of which DI framework/technique you are using. The Contracts for interfaces are a feature of the Code Contracts for .Net tool.

Interfaces are not Contracts

Interfaces are often referred to as Contracts in the context of IoC: they define the methods and properties that a service should implement and are a key factor in achieving the loose coupling: one can build the code against the interface and can plug in various implementation with no risks.

Unfortunately, interfaces are not contracts: while they specify how the methods signatures should be, interfaces do not specify the functional behavior. To illustrate this, let’s take a look at a well-known simple interface of the BCL:

For this interface, I could naively implement as follows:

While my implementation is really really wrong, it fulfills all requirements of the IServiceProvider interface: a method GetService that returns object. Of course, if I would have read the MSDN documentation of GetService, I would have known that object should implement serviceType. Unfortunately, the interface did not tell me anything about that and compilers don’t understand MSDN documentation either.

Contracts for interfaces

Code Contracts provides an API for design-by-contracts (pre-condition, post-condition, etc…). It also supports defining contracts for interface or abstract classes. Contracts for interfaces can be used to specify the functional behavior of interface members. While they also serve as documentation, those contracts can also be turned into runtime checks or leveraged by static checkers.

• since interface members cannot have method bodies, the contracts are stored in a ‘buddy’ type.
• The [ContractClass]/[ContractClassFor] attributes bind the interface type and the contract type together.
• Contract.Result<object>() is a helper method to refer to the result value, since this is not supported in C#.

Let’s take a closer look at the body of IServiceProviderContract.GetService. It contains a pre-condition (Requires) that the serviceType should not be null and a post-condition (Ensures) that the return value should be null or should be assignable to the serviceType:

In this simple case, the contracts captured precisely the specification that we found in MSDN. The critical difference is that they are stored in a format that compilers and runtimes know pretty well: MSIL byte code. The benefits are huge:

• documentation: the contracts are stored in a programming-language agnostic format that mined and rendered for your favorite programming language,
• static checking: static analysis tools can (and will) use contract to try to find bugs before you execute the code, or prove that it is correct with respect to the contracts,
• runtime checking: the runtime checker will instrument all implementations of the interface with the interface contracts automatically. Once you’ve specified how an interface should behave, you do not have to repeat yourself when re-implementing it, the re-writer takes care of that.
• automated white box testing: tools like Pex can leverage the runtime contract checks to try to find inputs that violate the contracts.
• IoC/DI framework agnostic: It does not matter which DI framework you use, as soon as you use interface, you could also provide contracts for it.

We’ve just released the Pex 0.13.40528.2 (Nikolai has all the details about the new features). In this release, a number of attributes were added to force Pex produce more friendly inputs.

The problem with Enums

When you apply Pex to code that uses Enumerations, you quickly learn something: the CLR does not constraints the enumeration values to be defined. This means than whenever you take an Enum as a parameter, you should validate it is properly defined:

void Test(SomeEnum e) {
    if (!Enum.IsDefined(typeof(SomeEnum), e)) throw new ArgumentException(“…”);
}

Look at the BCL code, they actually do this kind of validation. For good or bad, some of our users have been complaining that Pex should restrict the enumeration to defined values, in fact considering the generated failing test case as noise. To restrict generated enumartion values to be defined, you can add the [PexEnumValuesDefined(typeof(SomeEnum))] attribute on your test/fixture/assembly. Internally, this attribute attaches an invariant to the given enumeration type that tells Z3 the allowed values for that type.

What about multi-dimensional arrays?

Ever wakened a day wanting to iterate arrays starting from –12345? Well you can with multidimensional arrays… The Array.CreateInstance method has an overload that takes length and lower bounds to instantiate multidimensional arrays:

int[,] array = (int[,])Array.CreateInstance(typeof(int),
    new int[] {10, 10}, // lengths
    new int[] {–12345, – 789} // lower bounds
    );
a[-12345, –789] = –1;

Of course, this feels a bit weird because most languages won’t allow to create such arrays, but the Array.CreateInstance method is still available nonetheless. It feels a bit weird but this behavior is perfectly valid from the runtime point of view. To restrict Pex from generating such bounds, you can add the [PexZeroMdArrayLowerBounds(0)] attribute (one attribute per dimension).

What about Booleans?

Booleans is another interesting citizen of the CLR. At the MSIL level, the Boolean type is really a byte, widened as an integer when on the stack. Compilers try really hard to keep Booleans as 0 or 1 but… it is possible to craft Boolean values anywhere the byte range using safe MSIL code (not from C# though or any BCL method I know of). The implication of this are quite disturbing, for example, consider the following parameterized test:

[PexMethod] {
void BooleanMadness(bool a, bool b) {
     if (a && b && a != b)
         throw new Exception(“this is madness!”);
}

Clearly, in a word of Booleans constrained to 0 or 1, the branch that throws the exception should be unreachable. If you execute Pex on this test, it will generate 4 test one of which raises the exception:

Notice the actual values (0x02, 0x03) of the Booleans in the pretty printing. If you want Pex to generate Boolean that are restricted to 0 or 1, use the [PexBooleanAsZeroOrOne] attribute. This attribute forces Z3 to pick boolean values as 0 or 1. Btw, there’s already a connect bug for this behavior.

If you’ve been thinking about presenting Pex to your co-workers or your local .NET community, you can use our slide decks at http://research.microsoft.com/en-us/projects/pex/documentation.aspx . The slide decks are there to help you, don’t hesitate to shuffle them, cut them or pick whatever you need in them (and of course, tell us about it).

Cheers, Peli.

If you are (or you've been) blogging about Pex, let us know and drop the link in this following forum thread:

http://social.msdn.microsoft.com/Forums/en-US/pex/thread/7e6c26df-ccfb-48a4-878d-b3b01f5ba8d5

We are building a list of links that will be hosted our project web site.