# Friday, November 26, 2004

Question: So you are using files in your application, how do you test easily how it behaves if the file access is denied ?
Answer: Use declarative permissions to deny all access on the files

Ok that's a rough introduction, let me develop the idea. When your application relies on using resources such as files, TcpChannels, registry, etc... you should test how it behaves when the access to those resources is refused. The resource denial can be caused by a lot of external factors (loss of connectivity, etc...), lack of physical resources, .... or by denied security permissions. In other words, security permissions is an easy way to simulate various situations such as file access error, etc... 

Let me illustrate this by implementing a test decorator for MbUnit that makes all file access diened.

Creating DenyFileIOAccess decorator

As all test decorators, we derive a new attribute class from MbUnit.Core.DecoratorPatternAttribute and a new IRunInvoker from DecoratorRunInvoker:

using System;
using System.Security.Permissions;
using System.Collections;
using MbUnit.Core.Invokers;
using MbUnit.Core.Framework;
namespace MbUnit.Framework.Security
{
    [AttributeUsage(AttributeTargets.Class | AttributeTargets.Method,AllowMultiple  = false, Inherited =true)]
    public sealed class DenyFileIOAccessAttribute : DecoratorPatternAttribute
    {
        public override IRunInvoker GetInvoker(IRunInvoker invoker)
        {
            return new DenyFileIOAccessRunInvoker(invoker, this);
        }
        private sealed class DenyFileIOAccessRunInvoker : DecoratorRunInvoker
        {
            private DenyFileIOAccessAttribute attribute;
            public DenyFileIOAccessRunInvoker(
                IRunInvoker invoker,
                DenyFileIOAccessAttribute attribute)
                :base(invoker)
            {
                this.attribute = attribute;
            }
            [FileIOPermission(SecurityAction.Deny, AllFiles = FileIOPermissionAccess.AllAccess)]
            public override object Execute(Object o, IList args)
            {
                return this.Invoker.Execute(o, args);
            }
        }
    }
}

The important thing to notice in the snippet is the FileIOPermission attribute that tags the execute method. All the code executed inside that method will have file IO denied. So since Invoker.Execute calls the test method, we have what we are looking for.

DenyFileIOAccessAttribute in action

Let put the new decorator in action. The following snippet shows a simple test that tries to create a new file. The execution log using TestDriven.NET is outputed below. As one can see, a SecurityException was raised by the framework when trying to open the file.... it's magic.

using System;
using System.IO;
using MbUnit.Framework;
using MbUnit.Framework.Security;
namespace MbUnit.Demo
{
    [TestFixture]
    public class SecurityTest
    {
        [Test]
        [DenyFileIOAccess]
        public void SecureMethod()
        {
            using (StreamWriter writer = new StreamWriter("test.txt"))
            {
                writer.Write("we should not be here");
            }
        }
    }
}

-- output

------ Test started: Assembly: MbUnit.Tests.1.1.dll ------
Info: Test Execution
Info: Exploring MbUnit.Tests.1.1, Version=1.0.1791.9801, Culture=neutral, PublicKeyToken=null
Info: MbUnit 2.22.0.0 Addin
Info: Found 1 tests
Info: [failure] SecurityTest.SecureMethod
TestCase 'SecurityTest.SecureMethod' failed: Request for the permission of type 'System.Security.Permissions.FileIOPermission, mscorlib, Version=2.0.3600.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed.
 System.Security.SecurityException
 Message: Request for the permission of type 'System.Security.Permissions.FileIOPermission, mscorlib, Version=2.0.3600.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed.
 Source: mscorlib
 StackTrace:
 at System.Security.CodeAccessSecurityEngine.Check(PermissionToken permToken, CodeAccessPermission demand, StackCrawlMark& stackMark, Int32 checkFrames, Int32 unrestrictedOverride)
 at System.Security.CodeAccessSecurityEngine.Check(CodeAccessPermission cap, StackCrawlMark& stackMark)
 at System.Security.CodeAccessPermission.Demand()
 at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)
 at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy)
 at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
 at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize)
 at System.IO.StreamWriter..ctor(String path)
 \securitytest.cs(22,0): at MbUnit.Demo.SecurityTest.SecureMethod()
0 succeeded, 1 failed, 0 skipped, took 0.00 seconds.

---------------------- Done ----------------------

This methodology applies to the other types of permissions that can be found in the System.Security namespace.

Where can I get those ?

MbUnit will soon contain a few other of those attributes. Since it is a .Net 1.1 feature, they will be enclosed in the MbUnit.Framework.1.1.dll under the MbUnit.Security namespace.

posted on Friday, November 26, 2004 3:07:00 AM (Pacific Standard Time, UTC-08:00)  #    Comments [3]
# Tuesday, November 23, 2004

Reflector.CodeMetrics is a new Addin for Reflector that analyses and computes several code quality metrics on your assemblies. This addin uses the excellent Reflector API to compute classic metrics such as the cyclomatic complexity or more straightforward such as the number of local variables in a method. All results can be dumped to file (using DataSet.Save)

Download Reflector.CodeMetrics (look for Reflector.CodeMetrics.zip) For installation and usage instructions, please read the Readme.txt enclosed in the zip file.

Tip: when you explore the result, you can doubleclick on the border of the row to show the item in the decompiler

 

posted on Tuesday, November 23, 2004 12:21:00 PM (Pacific Standard Time, UTC-08:00)  #    Comments [14]
# Wednesday, November 17, 2004

This is a preview of a new Reflector Addin that computes various CodeMetrics on .Net assemblies. The excellent Reflector API makes it very easy to do this kind of things. Here are the metrics that I plan to implement :

  • Counting metrics: Module/Assembly, Type/Module, Method/Type, etc...
  • IL counting metrics: IL/Method, LocalVariables/Method, ExceptionHandlers/Method, etc...
  • IL flow graph metrics: components, cyclomatic complexity,
  • Method graph: methodrank, callers, callees, components, distance from entry point,
  • Type graph: type rank, inherance level, etc...

I have not done an extensive research on the subject so I might forget a lot of them, suggestions and links welcome.

Note that this uses a plugin architecture so that anyone can write and use his own metric.

Screenshot:

posted on Wednesday, November 17, 2004 6:50:00 PM (Pacific Standard Time, UTC-08:00)  #    Comments [18]
posted on Wednesday, November 17, 2004 2:55:00 PM (Pacific Standard Time, UTC-08:00)  #    Comments [0]
# Monday, November 15, 2004
posted on Monday, November 15, 2004 2:48:00 PM (Pacific Standard Time, UTC-08:00)  #    Comments [3]
# Wednesday, October 20, 2004

This new functionality enables you to specify a tabular list of test cases that are to be feeded to the test method. Let me illustrate that with the floating point division testing (as in the FIT framework Simple Example ):

numerator denominator quotient()
1000 10 100.0000
-1000 10 -100.0000
1000 7 142.85715
1000 .00001 100000000
4195835 3145729 1.3338196

You can now translate this directly to C# in MbUnit using the RowTestAttribute and the RowAttribute:

[TestFixture]
public class DivisionFixture
{
    [RowTest]
    [Row(1000,10,100.0000)]
    [Row(-1000,10,-100.0000)]
    [Row(1000,7,142.85715)]
    [Row(1000,0.00001,100000000)]
    [Row(4195835,3145729,1.3338196)]
    public void DivTest(double numerator, double denominator, double result)
    {
        Assert.AreEqual(result, numerator / denominator, 0.00001 );
    }
}

Of course, these tests are not very well targeted because we do not test the “special” floating point values such as 1,0,double.MaxValue, double.MinValue, NaN but you get the picture.

What if a test should throw ? In that case, you can specify the exception type as an additional parameter in the RowAttribute constructor:

    [Row(1,0,0, ExpectedException = typeof(ArithmeticException))]
    public void DivTest(double numerator, double denominator, double result)
    {...}

The final output of the tests is as follows where you can see that 5 tests (one per row) were generated and executed.

Info: Found 5 tests
Info: [assembly-setup] success
Info: [success] RowTestDemo.DivTest(0)
Info: [success] RowTestDemo.DivTest(1)
Info: [success] RowTestDemo.DivTest(2)
Info: [success] RowTestDemo.DivTest(3)
Info: [success] RowTestDemo.DivTest(4)
Info: [assembly-teardown] success
Info: [reports] generating HTML report
posted on Wednesday, October 20, 2004 7:30:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [7]
# Monday, October 18, 2004

I'm slowly catching up with putting some order in the MbUnit documentation. Therefore, MbUnit has now a Wiki (FlexWiki powered) part of the TestDriven.NET wiki. Feel free to contribute...

http://www.testdriven.net/wiki/default.aspx/MyWiki.MbUnit

posted on Monday, October 18, 2004 5:36:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [12]
# Saturday, October 16, 2004

MbUnit has now a basic support for loading fixture and tests based on their names, such fixture are called Naked Fixtures . Although this may look like a step backward in functionality, this can be handy in particular situations. For example, if you don't want to reference any test framework, this is a good solution because it does not require any type or attribute.

Naming rules

  • class name must end with “Fixture“
  • fixture setup method must be named “TestFixtureSetUp“
  • fixture teardown method must be name “TestFixtureTearDown“
  • setup method must be named “SetUp”
  • teardown method must be named “TearDown”
  • test methods must end with “Test”
  • all rules are case sensitive

Limitations

  • [Update: decorators are supported].
  • no combinatorial tests

QuickStart

 Below are two fixture. The first uses attributes and should be familiar to you. The second is it's naked equivalent:

[TestFixture]
public class ClassicFixture
{
    [TestFixtureSetUp]
    public void TestFixtureSetUp()
    {
        Console.WriteLine("TestFixtureSetUp");
    }
    [SetUp]
    public void SetUp()
    {
        Console.WriteLine("SetUp");
    }
    [Test]
    public void First()
    {
        Console.WriteLine("Test1");
    }
    [Test]
    public void SecondTest()
    {
        Console.WriteLine("Test1");
    }
    [TearDown]
    public void TearDown()
    {
        Console.WriteLine("TearDown");
    } 
    [TestFixtureTearDown]
    public void TestFixtureTearDown()
    {
        Console.WriteLine("TestFixtureTearDown");
    }
}

Naked version:

public class NakedFixture
{
    public void TestFixtureSetUp()
    {
        Console.WriteLine("TestFixtureSetUp");
    }
    public void SetUp()
    {
        Console.WriteLine("SetUp");
    }
    public void FirstTest()
    {
        Console.WriteLine("Test1");
    }
    public void SecondTest()
    {
        Console.WriteLine("Test1");
    }
    public void TearDown()
    {
        Console.WriteLine("TearDown");
    }
    public void TestFixtureTearDown()
    {
        Console.WriteLine("TestFixtureTearDown");
    }
}
posted on Saturday, October 16, 2004 10:43:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [1]
# Friday, October 15, 2004

TestDrivent.NET, which contains MbUnit, QuickGraph, TestFu and Refly, has gone RC2 .

See http://www.testdriven.net/wiki/default.aspx/MyWiki.DownLoad

 

posted on Friday, October 15, 2004 2:37:00 AM (Pacific Daylight Time, UTC-07:00)  #    Comments [4]
# Thursday, October 14, 2004

Check out this new Reflector addin that creates diffs. Very cool...

http://codingsanity.blogspot.com/2004/10/reflector-diff.html

posted on Thursday, October 14, 2004 5:14:00 PM (Pacific Daylight Time, UTC-07:00)  #    Comments [7]