# Thursday, December 04, 2008

I just released a new version of QuickGraph. Among many small improvements here and there, the big news are:

  • .net 2.0 support is back! QuickGraph now builds both for .net 2.0 and .net 3.5 (with extension methods).
  • The Dijkstra algorithm use a Fibonacci Heap under the hood, which is way more efficient. Courtesy of Roy Williams!
  • better serialization support and other bug fixes.

Enjoy, Peli.

posted on Thursday, December 04, 2008 12:28:30 AM (Pacific Standard Time, UTC-08:00)  #    Comments [2]
# Tuesday, December 02, 2008

Very exciting, I just published my first video on channel9. In the future, I’ll posting more interviews about researchers and projects that happens in our group in Redmond. Expect to get more videos on Pex, Code Contracts, CHESS, etc… over there.

http://channel9.msdn.com/posts/Peli/The-RiSE-of-Research-in-Software-Engineering/

posted on Tuesday, December 02, 2008 12:53:26 PM (Pacific Standard Time, UTC-08:00)  #    Comments [0]
# Tuesday, November 25, 2008

We just published a tutorial on the code digger on CodeProject, enjoy…

http://www.codeproject.com/KB/testing/pex.aspx

posted on Tuesday, November 25, 2008 8:13:29 AM (Pacific Standard Time, UTC-08:00)  #    Comments [0]
# Friday, November 14, 2008

We will give a session on Pex at the Seattle Code Camp – Saturday 11/15 at 2.45pm. We will show the new Code Digger and how Pex can help you…

posted on Friday, November 14, 2008 1:35:49 PM (Pacific Standard Time, UTC-08:00)  #    Comments [0]
# Wednesday, November 05, 2008

Check out the session on Code Contracts and Pex on Channel 9. You will learn about the new cool API to express pre-conditions, post-conditions and invariants in your favorite language – i.e. design by contracts (DbC) for .NET and the new Code Digger experience in Pex, and most importantly how DbC and Pex play well together.

http://channel9.msdn.com/pdc2008/tl51/

image

posted on Wednesday, November 05, 2008 11:10:22 PM (Pacific Standard Time, UTC-08:00)  #    Comments [4]
# Saturday, October 25, 2008

 

I’ll be at the MSR@PDC booth the entire week, don’t forget to say hello and get your demo.

image

posted on Saturday, October 25, 2008 7:29:17 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [0]
# Tuesday, October 21, 2008

Wonder what we’ve been up to for the last months… We’ve been building a very cool development experience on top of Pex that we call Code Digging. Check out Nikolai’s blog post on what it means to you!       .

image

posted on Tuesday, October 21, 2008 9:49:02 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [0]

Have you ever written code that directly used the .NET File APIs? We probably all did although we knew it would make the code less testable and dependent on the file system state. As bad as it sounds, it really requires a lot of discipline and work to avoid this: one would need to create an abstraction layer over the file system, which is not a short task (think long/tedious).

// in how many ways can this break?
public static void CleanDirectory(string path)
{
    if (Directory.Exists(path))
        Directory.Delete(path, true);
    Directory.CreateDirectory(path);
}

Abstraction

Fortunately, there always someone else who got motivated at some point. Ade Miller digged an abstraction of the File System, the IFileSystem interface, that Brad Wilson had written for the CodePlex client project. Very nice since it provides a solid foundation for cleanly abstracting from the File System, and thus increase the testability of our code.

// a little better, testable code at least
public static void
CleanDirectory(IFileSystem fs, string path) { if (fs.DirectoryExists(path)) fs.DeleteDirectory(path, false); fs.CreateDirectory(path); }

Mocking

So with this interface we can write code that we’ll be able to test in isolation from the physical file system. That’s great but there is still a lot of work on the should of the developer: the developer will have write intricate scenarios involving mocks to simulate the different possible configurations of the file system. No matter which mock framework (Moq, Rhino, Isolator, …), he’ll be using, (1) it’s going to be painful, (2) he’ll miss cases. It’s probably easy to write a single “happy path”, but especially with the file system there are quite some realistic “unhappy paths”.

This test case uses Moq to create the scenario where there is a directory already. Although Moq has a very slick API to set expectations, it is still a lot of work to write this basic scenario. (And what exactly is the meaning of “Expect”, the delegate or expression inside, “Returns” and “Callback”?)

[TestMethod]
public void DeletesAndCreateNewDirectory()
{
    var fs = new Mock<IFileSystem>();
    string path = "foo";

    fs.Expect(f => f.DirectoryExists(path)).Returns(true);
    fs.Expect(f => f.DeleteDirectory(path, false)).Callback( () => Console.WriteLine("deleted"));
    fs.Expect(f => f.CreateDirectory(path)).Callback(() => Console.WriteLine("created"));

    DirectoryExtensions.CleanDirectory(fs.Object, path);
}

Modeling

We had our intern, Soonho Kong, work on a Parameterized Model of the File System, built on top of the IFileSystem interface (yes that same interface Brad Wilson published on CodePlex). We say that the model is parameterized because it uses the Pex choices API to create arbitrary initial File System states; Pex “chooses” each such state (actually, Pex carefully computes the state using a constraint solver) to trigger different code paths in the code. You can think of each choice as a new parameter to the test. Or to put this with an example: if your code checks that the file “foo.txt” exists, then the parameterized model would choose a file system state that would contain a “foo.txt” file (or not, in another state, to cover both branches of the program).

So what does it mean for you? Well, the way you write tests that involve the file system changes radically. You simply need to pass the file system model to your implementation. The model is an under-approximation of the real file system (which means that we didn’t model every single nastiness that can occur when the moon is full), but it definitely captures more practically relevant corner cases than we (humans) usually think about. Let’s see this in the following test:

[PexMethod]
public void CleanDirectory()
{
    var fs = new PFileSystem();
    string path = @"\foo";
    try
    {
        DirectoryExtensions.CleanDirectory(fs, path);

        // assert: the directory exists and is empty
        Assert.IsTrue(fs.DirectoryExists(path));
        Assert.AreEqual(0, fs.GetFiles(path).Length);
    }
    finally
    {
        fs.Dir();
    }
}

When we run Pex, we get 7 generated tests. In fact, Pex finds an interesting bug that occurs when a file with the name of the directory to clean already exists. In the Pex Exploration Results window, you can see a ‘dir’-like output of the file system model associated with a particular test case (the fs.Dir() method call outputs that text to the console which Pex captures).

image

This bug is the kind of corner-case that makes testing the file system so fun/hard. Thanks to the parameterized model (and Soonho), we got it for free. Note also that the assertion in our test is pretty powerful since it must be true for any configuration of the file system (it almost smells like a functional specification to me):

// assert: the directory exists and is empty
Assert.IsTrue(fs.DirectoryExists(path));
Assert.AreEqual(0, fs.GetFiles(path).Length);

Happy modeling!

The full source of PFileSystem will be available in the next version of Pex (0.8)..

posted on Tuesday, October 21, 2008 12:15:09 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [6]
# Friday, October 17, 2008

In this post, you’ll find the XSL stylesheets to render MSTest 2008 reports to CruiseControl.NET. CC.NET comes with stylesheets for MSTest 2005 but it seems that there were some changes in the xml output or at least they did not work for me. Please refer to the CC.NET documentation on how to integrate them into your build.

The templates are minimalistic since a deeper investigation can be done by importing the .trx file into Visual Studio.

The summary view:

image

And the details view:

image

Downloads – "AS IS" with no warranties, and confers no rights – :

posted on Friday, October 17, 2008 10:28:51 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [0]
# Monday, October 13, 2008

In any unit testing activity, there always comes a point when one needs to isolate the code under test from the environment. This usually involves adding interfaces at the environment boundaries, which can be used to test the code in isolation. Because implementing those interfaces ‘by hand’ for the test can be tedious, an ecosystem of mock frameworks has flourished (NMock, Rhino Mocks, Moq, there’s more…). In fact, it’s a pretty competitive area.

The dark side of Mocks frameworks

Under their friendly API, mock frameworks usually hide a complex machinery: dynamic code (through Reflection.Emit or a rewriting profiler), expression trees or reflection.

From a Pex point of view, this ‘machinery’ puts a large burden on the dynamic whitebox code analysis; Pex does a precise data flow and control analysis for any code that is executed, this means that it also has to analyze the mock infrastructure.  

Therefore, we decided to build a framework to define stubs that would try to avoid the overhead of dynamic code generation (just rely on source code) and still provide the same level of flexibility of existing mock frameworks: defining the behavior of the stub inside the test, strongly typed, intellisense support, first-class debugging experience, etc…

Low tech Stubs using delegates

With C# 3.0 lambdas, it is quite easy to define delegates. Therefore, delegates seemed an elegant and simple way for building stub methods (we’re not the first ones to think about this). For example, a stub of the IServiceProvider would simply be

class SServiceProvider : IServiceProvider {
    // this field can be set to ‘override’ the behavior of GetService
    public Func GetServiceHandler;

    object IServiceProvider.GetService(Type type) {
        var d = this.GetServiceHandler;
        if (d != null) return d(type); // i’m stubbed :)
        else throw new NotImplementedException();
    }
}

Enter the Stubs framework

We took this simple idea and turned it into the Stubs framework. The Stubs framework comes as a enhanced single file generator for Visual Studio which generates stubs for selected assemblies. Special attention was paid to make sure the code generation did not get into the feet for the unit test writer (we track the build events and regenerate when needed).

More about stubs can be found at http://research.microsoft.com/pex/stubs.pdf. This is a preview of what’s coming up on the next release of Pex.

Happy stubbing.

posted on Monday, October 13, 2008 8:07:23 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [1]