If you have the chance to attend Tech Europe, don’t forget to attend our session on Code Contracts and Pex . Nikolai Tillmann and I will also be attending a booth (on Code Contracts and Pex) during the week. See you there.

DEV315 Code Contracts and Pex: Power Charge Your Assertions and Unit Tests
Friday 11/13 – 13:00pm – 14:15pm.

We have just release a new version of Pex (v0.17.xxx) which brings new features for Moles and more bug fixes.

Per-Instance Moles

Instance methods may be moled differently per object instance. This is particularly useful when multiple instances of same type need to behave differently in the same test. For example, let’s consider a simple class Bar that has a ‘Run’ method to check that the values are not the same. The important point here is that we would not be able to deal with such cases if we could not mole the behavior on a per-instance basis.

In the test case of Run, we create to instance of MBar (the mole type of Bar) and assign two different delegates to the ‘Value’ getter. Moles are active as soon as they are attached (unlike the previous version) so we can just call the ‘Run’ method once the moles are set up. In the code below, we are using the C# Object Initializers feature that allows us to set properties of a newly created object within curly braces after the ‘new’ call.

As you may have noticed, we just wrote a parameterized unit test that takes arbitrary ‘left’ and ‘right’ values. We then simply execute Pex to find the relevant values to cover the ‘Run’ method:

Moles for Constructors

In same cases, objects created inside of other methods need to be moled to test the actual program logic. With moles you can replace any constructor with your own delegate, which you can use to attach a mole to the newly created instance. Let’s see this with another simple example where a  ‘Run’ method creates and uses an instance of a ‘Foo’ class.

Let’s say we would like to mole the call to the ‘Value’ property to return any other value. To do so, we would need to attach a mole to a future instance of Foo, and then mole the Value property getter. This is exactly what is done in the method below.

We then run Pex to find that 2 different values are needed to trigger all code paths.

Mole interface binding

All the member implementations of an interface may be moled at once using the new ‘Bind’ method. This smells like duck typing with type safety as the Bind methods are strongly typed. For example, we want to mole a collection type (Foes) to return a custom list of Foo elements (which need to be moled too). The goal is to test the sum method…

In the parameterized unit test case, we create an instance of Vector, which implements IEnumerable<int>. Then, we can simply bind the ‘values’ array to the mole to make Vector iterate over the array. The call to Bind will mole all methods of MVector that are implemented by the ‘values’ parameters, effectively redirecting all the enumeration requests to the ‘values’ array.

When Pex runs through the sample, it correctly understand the data flow of the parameters through the moles and finds inputs that break the ‘sum >= 0’ assertion:

Compilation of Stubs assemblies

When one needs stubs or moles for a system assembly, it does not really make sense to re-generate the stubs each time the solution is loaded. To that end, Stubs ships with a command line tool that lets you easily compile a stubs assembly into a .dll:

stubs.exe mscorlib

Once the stubs assembly is compiled, i.e. mscorlib.Stubs.dll is created, you simply have to add a reference to it in your test project to start using it. In future versions of Pex, we will provide a better experience to support this scenario.

Bug Fixes

• A number of fixes from the issues reported through the forums and bugs reported to pexbug@microsoft.com .

Breaking Changes

• The code generation of Moles was significantly changed. This might mean that you will have to recompile your solution, and adapt all existing uses of Moles.

Phil Haack recently wrote a blog post about a tricky use of Moq where one needs to set expectations for successive method calls. Here’s the specification of the problem shamelessly copied from his blog post:

So we want to set up a mock to return 0 or more ‘true’ and a final false. With Moq and the help of an extension method, Phil managed to express this in a single line of C#.

Let’s do it with Stubs!

Stubs is a stub framework solely based on delegates. In that sense, it does not provide any helper to express a sequence of calls so we need to do that in C#. An easy approach is to create an array and use a counter to iterate through it (sounds familiar right?). This is exactly that this snippet does:

SIDataReader is the code generated stub type of IDataReader. It has a field Read of type Func<bool> that can used to stub the Read() method.

Putting Pex into the mix

Why did we have to specify the values in the test? We should let Pex decide the sequence of booleans that are relevant to test code. To do so, we refactor values as a parameter and add an assumption to ensure that the last element is false.

Voila!

We just released v0.16.40915.5 of Pex. The major highlights of this release are Moles, a lightweight detour framework, and better support for test frameworks.

• Moles, a lightweight detour framework: Moles is a new extension of the Stubs framework: it lets you replace any .NET method (including static methods) with your own delegate.(sample outdated) 
  Want read more about this sample? here is the full walkthrough.
  The moles framework is not yet feature complete; moles does not support constructors and external methods. Some types of mscorlib cannot be moled as they interact too deeply with the CLR.
• Pex gets its own HostType: A HostType is a feature for the Visual Studio Unit Test framework that lets specific unit tests be run under specialized hosts such as Asp.Net or VisualStudio iself. In order to create reproducible test cases using Moles, we had to implement a HostType that lets tests be run under the Pex profiler. This is very exciting because it also opens the door for many uses of Pex such as fault injection, dynamic checking, etc… in future versions of Pex. When generating test cases with Pex, all the necessary annotations will be created automatically. To turn the Pex HostType on hand-written (non-parameterized) unit tests, simply add [HostType(“Pex”)] on your test case.
   
  This feature only works with Visual Studio Unit Test 2008.
• Select your test framework: the first time you invoke ‘Pex’ on your code, Pex pops up a dialog to select your test framework of choice. You can select which test framework should be used by default and, more importantly, where it can be found on disk.
  
  If you do not use any test framework, Pex ships with what we call the “Direct test framework”: in this “framework”, all methods are considered as unit tests without any annotation or dependencies.
   
  These settings are stored in the registry and Pex should not bug you again. If you want to clear these settings, go to ‘Tools –> Options –> Pex –> General’ and clear the TestFramework and TestFrameworkDirectory fields:
    
   
• Thumbs up and down: We’ve added thumbs up and down buttons in the Pex result view. We are always looking for feedback on Pex, so don’t hesitate to click them when Pex deserves a thumbs up or a thumbs down.
  
• Performance: Many other performance improvements under the hood which should avoid Pex hog too much memory in long running scenarios.
• Miscellanous improvements and bug fixes:
  • Support for String.GetHashCode(): we now faithfully encode the hashing function of strings, which means Pex can deal with dictionaries where the key is a string.
  • Fewer “Object Creation” messages that are not actually relevant.
  • In VS 2010, the package used to trigger an exception when the solution loaded. This issue is now fixed.
  • In Visual Studio when an assembly had to be resolved manually (i.e. that little dialog asking you where an assembly is), Pex remembers that choice and does not bug you anymore.
  • And many other bugs that were reported through the forums.

Since it’s been almost a year that we’ve released Pex, we have started a little survey on our forums. We are looking for feedback on how you use Pex and how you would like to use it. If you are interested, please voice yourself  on our mini survey at

http://social.msdn.microsoft.com/Forums/en/pex/thread/27c703ea-3928-41ef-b767-638c39966b99

The questions are:

1. Are you using Pex already? A) From Visual Studio, B) the command-line, C) other, D) no
2. Which version of Visual Studio are you using? A) Professional, B) Team System, C) other
3. How many people are in your team?
4. Do you write A) Parameterized Unit Tests (PUTs), or B) only use the Wizard-generated PUT stubs?

Any other comments are appreciated as well, about the license terms, or required Visual Studio versions…
Thanks,  The Pex Team

While on parental leave, I could sneak a couple hours to read a paper on Frontier Search by Korf and al. Frontier search is a kind of best-first search algorithm that reduces the needs for memory. A very interesting and relaxing read… I add an implementation to QuickGraph that follows the idea.

The new release of Code Contracts is out and brings a very cool feature: Xml comment generation. This means that you do not have to worry about keeping the comments in sync with the code, the compiler takes care of this.

Contracts to Xml Comments in action

Unless you invest a lot of work in them, Xml comments are most often worthless. For example, in QuickGraph, the documentation of the Edge constructor is really… well… useless (oops my fault):

However, the body of the constructor contains Contracts that state the pre-conditions and post-conditions of the constructor: source and target should not be null, etc… With the new xml comment generation, these contracts will be added to the xml documentation and ultimately will show up in the compiled documentation.

First, let’s turn xml comment generation on in the property pages:

Once the project is rebuilt, we can take a look at the generated xml documentation file. The ‘requires’ and ‘ensures’ elements are now under the Edge constructor element:

Finally, we run the documentation file through Sandcastle*** to get the final result (make sure you update the Sandcastle stylesheets that come with Contracts. Read section 8 of the documentation):

Voila! With this feature, you have a documentation that never goes out of date.

I’ve just made a release of QuickGraph 3.3 on CodePlex. Along with the usual bug fixes, this version brings a set of new graph data structures based on delegates. Instead of taking care of the storage, these graphs simply ‘query’ delegates for vertices and out-edges. You can use them to bridge any existing graph data structure in your code with QuickGraph – with almost no performance/memory penalty. (Delegate graphs can also be used to deal with graphs that won’t hold in memory but that’s another story).

Let’s see this with an example. Suppose, we have a simple graph representation using a jagged array, int[][] graph, where each edge is defined as (i, graph[i][j]) :

In order to use this graph with QuickGraph, we ‘wrap’ it up into a delegate graph. There are a number of extension methods in GraphExtensions that take care of the common cases. Delegate graphs rely on 2 delegates: one that returns the vertices, the other one out-edges:

At this point, we can apply any of the algorithms that QuickGraph provides. For example, computing a topological sort of the vertices:

The new delegate graphs are ready to be downloaded at http://quickgraph.codeplex.com/Release/ProjectReleases.aspx .

Happy programming!

We just released a new version of Pex, v0.15.40714.1. This version does not contain any new features but… it brings better memory management and bug fixes for issues that were reported through our MSDN forums. Happy testing!

It contains a couple breaking changes too:

• Microsoft.Pex.Stubs.dll does not exist anymore and has been integrated into Microsoft.Pex.Framework.dll
• PexExplorableFromFactoriesAttribute renamed to PexExplorableFromFactoriesFromAssemblyAttribute

We have just released a new version of Pex, v0.14.040610.2. This version brings a range of Bug Fixes that were reported on the MSDN forums and a couple new features: Fix Suggestions for Invariant Methods and Natural properties  for Stubs.

Fix Suggestions for Invariant Methods

When you use Code Contracts, then Pex can now help you to add missing invariants, just as Pex already inferred missing preconditions in the past. Take for example the ArrayList class that comes with the samples of our recently updated tutorial slide deck. This class has the following fields:

When you add an empty invariant method,

and enable Runtime Checking of contracts, then Pex will create instances of this class using reflection while making sure that the invariants holds (we announced this feature in the last release). However, with an empty invariant method, Pex will be able to create illegal states, e.g. where the _items array is null, which may cause a NullReferenceException, and other issues. To get the following test cases, we let Pex explore the Add method of our ArrayList class:

When you look at the details of the failing test cases, then you will see that Pex now offers the option to add invariants directly into your [ContractInvariantMethod]:

Natural Properties for Stubs

The logic to handle properties with setters and getters generated by Stubs has been slightly improved so that it is possible to force properties to have as if they had a backing field.

When a getter or a setter of a property is invoked, Stubs checks whether the getter delegate or the setter delegates have been set. If not, i.e. no custom user behavior, it queries the fallback behavior for an initial value of the property. If the fallback behavior successfully returns a value, Stubs generates a closure and binds the getter and setter.

For example, here’s a simple IFoo interface with a Bar property:

One could write the following parameterized unit test:

Since we expect the Bar property to behave as it was wrapping a field, we don’t expect the assertion to be raised. When we execute this test with Pex, and the Pex fallback behavior, Pex will choose a value for ‘Bar’, and this value will be ‘stored’ the property. Therefore, we get as expected 2 test cases:

Non default constructor support for Stubs

In previous versions, Stubs could not generate stubs for classes with no default constructor. This is no longer the case, Stubs will generate one public constructor for each public or protected base constructor.

Bug Fixes

• Various code generation tweaks to make StyleCop happier
• Visual Studio addin crashes when loading in VS2010
• Visual Studio addin does not find attributes when they are globally qualified
• Added missing substitutions for System.Threading.Interlocked members
• Stubs automatically imports missing references
• When an object has an invariant method (from Code Contracts), Pex would emit assertions using the private fields
• Editorial issues to patterns paper
• Contracts no longer kill the process when the runtime rewriter was not executed (.net 4.0)
• Fixed bug in generic type guesser
• Fixed samples for the latest version of Stubs

As always, don’t hesitate to send us your feedback, good or bad, through our MSDN forums.