Saturday, February 23, 2008

So how do you write negative test cases with Pex? Here's a nice solution I was working on. I'm wondering

Traditional ExpectedExceptions style

Let's test the constructor a type Foo that takes a reference argument. We expect to throw a ArgumentNullException when null is passed. Therefore, we could write

[ExpectedException(typeof(ArgumentNullException))]
void Test() {
     new Foo(null);
}

or using xUnit style assertions,

void Test() {
     Assert.Throws(delegate { new Foo(null); });
}

Pex ExpectedExcetions style

Let's refactor our first test and push 'null' as a test parameter. We add an assertion after the constructor to make sure a null parameter never succeeds.

void Test(object input) {
    new Foo(input);
    Assert.IsNotNul(input); // we should never get here
}

The interesting part is that this test will not only test for the null value but also for the passing values as well. Moreover, there might be more checks over the input which might trigger other exceptions. In that case, additional asserts could be added -- or even better, centralized in an invariant method.

So, what do you think?

Tuesday, February 26, 2008 1:30:05 AM UTC
The assertion verifies that we do no reach the specified line but it says nothing about the reason it was not reached. Perhaps is was not reached due to ArgumentNullException (what I would hope) but it could just as well have been a NullReferenceException or something more exotic like a StackOverflowException caused by some really bad logic (although this would probably kill Pex, I imagine).

So if anything, I'd recommend using the 'Assert.Throws<ArgumentNullException>(delegate { ... })' style.
Tuesday, February 26, 2008 12:30:46 PM UTC
You have a point :)

This sample is not very precise. In fact, Assert.Throws is not that precise either when you think about it: the delegate could make some calls in the BCL which throw the ANE exception, bubbles up and finally passes the test.

Comments are closed.