Peli's Farm - Performance Testing using PerformanceCounters in MbUnit

Wednesday, July 28, 2004

Performance Testing using PerformanceCounters in MbUnit

The next version of MbUnit (2.20) will have a new custom attribute that will let you assert on the values of PerformanceCounters: PerfCounterAttribute. (Idea suggested by ISerializable). This means that you can make an assertion an every perfomance counter on your machine!

Here's a sample fixture that shows how the attribute can be used:

using System;
using MbUnit.Core.Framework;
using MbUnit.Framework;
namespace MbUnit.Demo
{
  [TestFixture]
  public class PerfCounterDemo
  {
    [Test]
    [PerfCounter(".NET CLR Memory", "% Time in GC", 10)]
    public void AllocateALotOfObjects()
    {
      ...
    }

    [Test]
    [PerfCounter(".NET CLR Loading", "% Time Loading", 10)]
    [PerfCounter(".NET CLR Security", "% Time in RT checks", 10000)]
    [PerfCounter(".NET CLR Security", "% Time Sig. Authenticating", 10)]
    [PerfCounter(".NET CLR Memory", "# Bytes in all Heaps", 5000000, Relative =true)]
    [PerfCounter(".NET CLR Jit", "% Time in Jit", 10)]
    public void MonitorMultipleCounters()
    {
      ...
    }
  }
}

Note that if you are too lazy to remember the names of the counters, I have written a CodeSmith template that creates a class filled with static helper methods for retreiving the counters. For example, the following class lets you write things like PerfCounterInfo.NetClrExceptions.NbofExcepsThrown.NextValue() using intellisense and without typing errors :)

using System;
using System.Diagnostics;
namespace MbUnit.Core.Framework
{
 public class PerfCounterInfo
 {  
  public sealed class NetClrExceptions
  {
   const string categoryName = @".NET CLR Exceptions";

   public sealed class NbofExcepsThrown
   {
    const string counterName = @"# of Exceps Thrown";

    public static float NextValue()
    {
     return NextValue(Process.GetCurrentProcess().ProcessName);
    }    
   }
   ...

posted on Thursday, July 29, 2004 5:29:00 AM UTC

Comments [8]
Tracked by:
http://www.google.com/search?q=mwinyplq [Pingback]
"100 debt elimination" (online) [Trackback]

Monday, June 06, 2005 5:41:43 PM UTC

Very nice.. How would I use the template to create the class? Does it only rely on <a title="CodeSmith" href="http://www.ericjsmith.net/codesmith/" target="_blank">CodeSmith</a>?
 
 Sorry for all the questions but all of your <a title="CodeSmith" href="http://www.ericjsmith.net/codesmith/" target="_blank">CodeSmith</a> templates confuse me.
 
 Thanks

Jerrad Anderson

Monday, June 06, 2005 5:41:44 PM UTC

Just double click on the template, select the Categories you want, and hit generate. Here's the latest source:
 
 <%@ CodeTemplate Language="C#" TargetLanguage="C#" Description="Perf diag template." %>
 <%@ Property Name="ClassName" Type="System.String" Default="PerfCounterInfo" Category="Options" Description="ClassName." %>
 <%@ Import Namespace="System.Diagnostics" %>
 <%@ Import Namespace="System.Text.RegularExpressions" %>
 
 using System;
 using System.Diagnostics;
 
 namespace <a title="MbUnit, Generating Unit Testing and Model Based Testing Framework for .NET Framework" href="http://mbunit.tigris.org" target="_blank">MbUnit</a>.Core.Framework
 {
 public class <%= ClassName %>
 {
 #region Constructors
 private <%= ClassName %>() {}
 #endregion
 
 #region Categories
 <% foreach(PerformanceCounterCategory cat in PerformanceCounterCategory.GetCategories()){
 if (!this.categories.Contains(cat.CategoryName))
 continue;
 try
 {
 PerformanceCounter[] counters=null;
 bool hasInstance = cat.GetInstanceNames().Length!=0;
 if (!hasInstance)
 counters = cat.GetCounters();
 else
 counters = cat.GetCounters(cat.GetInstanceNames()[0]);
 foreach(PerformanceCounter counter in counters) {};
 %>
 #region <%= cat.CategoryName %>
 /// <summary>
 /// <%= cat.CategoryHelp %>
 /// </summary>
 public sealed class <%= GetCategoryClassName(cat) %>
 {
 public const string CategoryName = @"<%= cat.CategoryName %>";
 #region Constructors
 private <%= GetCategoryClassName(cat) %>(){}
 #endregion
 
 <%foreach(PerformanceCounter counter in counters) {%>
 #region <%= counter.CounterName %>
 /// <summary>
 /// <%= counter.CounterHelp %>
 /// </summary>
 public sealed class <%= GetCounterClassName(counter)%>
 {
 public const string CounterName = @"<%= counter.CounterName%>";
 private <%= GetCounterClassName(counter)%>(){}
 <% if (hasInstance){%>
 /// <summary>
 /// Gets the value of the <see cref="PerformanceCounter"/>.
 /// <summary>
 /// <returns>
 /// Value returned by <see cref="PerformanceCounter.NextValue"/>
 /// for the current instance.
 /// </returns>
 public static float NextValue()
 {
 return NextValue(Process.GetCurrentProcess().ProcessName);
 }
 
 public static float NextValue(string instanceName)
 {
 return PerfCounterAssert.NextValue(
 <%= GetCategoryClassName(cat) %>.CategoryName,
 CounterName,
 instanceName
 );
 }
 <%}else{%>
 /// <summary>
 /// Gets the value of the <see cref="PerformanceCounter"/>.
 /// <summary>
 /// <returns>
 /// Value returned by <see cref="PerformanceCounter.NextValue"/>.
 /// </returns>
 public static float NextValue()
 {
 return PerfCounterAssert.NextValue(
 <%= GetCategoryClassName(cat) %>.CategoryName,
 CounterName,
 null
 );
 }
 <%}%>
 }
 #endregion
 <%} // GetCounters %>
 }
 #endregion
 <%
 }catch(Exception)
 {}
 %>
 
 
 <%} // GetCategories %>
 #endregion
 
 public static float NextValue(
 string categoryName,
 string counterName,
 string instanceName
 )
 {
 using (PerformanceCounter counter = new PerformanceCounter(
 categoryName,
 counterName,
 instanceName,
 true)
 )
 {
 return counter.NextValue();
 }
 }
 }
 }
 
 <script runat="template">
 private System.Collections.Specialized.StringCollection categories = null;
 
 [Category("Options")]
 public System.Collections.Specialized.StringCollection Categories
 {
 get
 {
 if (categories==null)
 {
 categories = new System.Collections.Specialized.StringCollection();
 foreach(PerformanceCounterCategory cat in PerformanceCounterCategory.GetCategories()){
 categories.Add(cat.CategoryName);
 }
 }
 return categories;
 }
 set
 {
 this.categories=value;
 }
 }
 
 public string Capitalize(string name)
 {
 return name[0] + name.Substring(1,name.Length-1);
 }
 public string Normalize(string name)
 {
 name = Regex.Replace(name,@"[A-Z]+",new System.Text.RegularExpressions.MatchEvaluator(Capit));
 name = Regex.Replace(name,@"#",@"Nb");
 name = Regex.Replace(name,@"\W",@"");
 
 return name;
 }
 
 public string Capit(System.Text.RegularExpressions.Match m)
 {
 return m.Value[0] + m.Value.Substring(1,m.Value.Length-1).ToLower();
 }
 public string GetCategoryClassName(PerformanceCounterCategory cat)
 {
 string name = Normalize(cat.CategoryName);
 
 return String.Format("{0}",name);
 }
 public string GetCounterClassName(PerformanceCounter counter)
 {
 string name = Normalize(counter.CounterName);
 
 return String.Format("{0}",name);
 }
 </script>

Jonathan de Halleux

Monday, June 06, 2005 5:41:44 PM UTC

Nice addition. Now we can replace the code we've written to check perf stats with WMI in unit tests. FYI: WMI validation might be another interesting attribute. It has some more stats than performance counters.
 FYI: Assert.AreEqual("Perfomance", "Performance");// Check title :)

Scott Willeke

Monday, June 06, 2005 5:41:45 PM UTC

Hi Scott,
 
 Any links on WMI, tutorial ?

Jonathan de Halleux

Monday, June 06, 2005 5:41:45 PM UTC

Peli's Blog

Monday, June 06, 2005 5:41:45 PM UTC

Peli's Blog

Monday, June 06, 2005 5:41:46 PM UTC

I posted some comments to a friend of mine who has been working with me on the perf testing about using WMI for such things: <a target="_new" href="http://blogs.pingpoet.com/philblog/archive/2004/06/16/592.aspx#594">http://blogs.pingpoet.com/philblog/archive/2004/06/16/592.aspx#594</a>

Scott Willeke

Monday, June 06, 2005 5:41:46 PM UTC

Greetings from Malaga (Spain). Antonio :-)

Malaga

Comments are closed.

Disclaimer
The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.