One of the topics of discussion at the XP workshop I went to on Tuesday was code metrics, and, subsequently, tools that could be used to obtain said metrics. Here’s a quick summary of some of the metrics we talked about:

CC (Cyclomatic Complexity) – Basically, this is the # of decision paths in a given method. More decision paths mean more possible points of failure and more required test scenarios.

Ca (Afferent Couplings) – This refers to the # of types directly depending on a particular other type. Types with lots of dependencies are considered to have high responsibility. I like to remember this one as the Type A Coupling – as in, a type with a high Ca has a Type A personality and is highly responsible.

Ce (Efferent Couplings) – This refers to the # of types a particular type directly depends on. A type that depends on lots of other types is highly dependent. I remember this one as the Everybody Coupling - as in, a type with a high Ce depends on Everybody. (just humor me ok)

I (Instability) – A # based on a ratio of Ca and Ce indicating an item’s resilience to change.

A (Abstractness) –  A ratio of the # of abstract types to types.

D (Distance) – This is a measurement of how far from the ideal I + A something is.

LOC (lines of code) – Uh, lines of code (although, surprisingly, this can cause controversy).

For more in depth discussion go here or here, or read Robert C. Martin’s book – Agile Software Development (Quick aside – this book is a MUST read for any developer. It is quite honestly the best software development book I have ever read. There. I’ve thrown down the gauntlet. If you haven’t read it, go get it right now. Tell’em I sent ya.)

While I’d read about these different coding metrics before, talking about them and then subsequently reading this blog post by Michael Swanson really showed me how useful these metrics might be in highlighting areas where problems might exist in a codebase, and spurred me into action to look into how I could integrate this analysis into our automated build cycle.

So I set out to research different code metrics tools. My main criterion:

1. At least hit the highlights such as Cyclomatic Complexity, Afferent Coupling, Efferent Coupling, and LOC.

2. Has to be Open Source and/or Free.

3. Has to be able to integrate into an automated build using NAnt (so, callable from the command line and able to spit out some output in the form of xml or some such).

Raymond had mentioned a tool called NDepend. In addition, I looked at Source Monitor based on Michael’s aforementioned blog post, and one named vil. (If anybody knows of more that meet the criteria, please let me know).

Source Monitor
What I liked
– LOC and CC analysis is based on the actual source code (rather than IL as some of the other tools). This, in my opinion, provides a more accurate assessment based on the original definitions of the metrics and makes it easier to judge the results based on industry recommendations.
– Examples showing how to integrate with a build (using Ant, so I’ll have to translate these to NAnt).
– Kiviat diagrams. An excerpt from Michael: 

The Kiviat diagram below is from some sample code I generated awhile back. I can quickly tell from this diagram that I should check to see if I've commented my code properly and take a look to see which method(s) are driving my complexity and depth outside typical limits (which are configurable). Fortunately, SourceMonitor makes it very easy to find and review the complex code.

What I Didn’t Like
– It looks like it may be fairly involved to get this going in my build script (although the Ant example will certainly help). Also, I’m not sure if I can get the Kiviat diagrams in my html reports yet - they may only be available in the client app as I didn’t see them in the example – which would be a bummer. I guess we will see. I will update this if this turns out not to be the case.

NDepend
What I Liked:
– Packs a lot of information in.
– Nice ability to configure which assemblies are included, and which are considered framework or outside of your control.
– Some really nice graphs such as the Assemblies Abstractness vs Instability and an Assemblies Dependencies Diagram (see below).



What I Didn’t Like:
– Analysis is based on IL rather than actual source files, so some of the numbers are a bit inflated – particularly the one for Cyclomatic Complexity, which the documentation suggest could be off by a factor of 2 or 3.
– No Configurability of what metrics get output as far as I could tell – so it’s all or nothing, which is ok, I suppose, but I’m not sure I want to use all the metrics at this point. I would’ve liked to have been able to specify the ones I’m interested in.
– Very little control of where the output gets sent to. 

vil
What I Liked:
– Highly configurable as to what types of things (i.e. classes, enums, structs, methods, etc) should and should not be analyzed, and what kinds of metrics should be output.
– Ability to filter the results based on criteria. The documentation is a little bit sparse on examples here, but I think this might allow someone to, say, spit out all methods that have a CC > 10, or something like that. If so, this could be really useful.
– A CruiseControl.NET xsl plugin has already been created for vil, along with an example of how to set it up in a NAnt script.

What I Didn’t Like:
– Again, analysis based on IL.
– I’m not sure if I’m not using it properly or what (although I tried almost every possible command switch combination), but when I ran the tool against some existing code that had approximately 50–60 classes, vil only displayed information about 4 of them. I plan to send an email to the support address and see if I can figure out what’s going on.

I decided that each had enough positives to warrant a shot, so I’d try to integrate each of them into my CruiseControl.NET/NAnt build and see how it went and see which ones were keepers. I’m about half way through this process now, so over the next few days I will post about my experiences with each. Stay tuned.

(Also, if anybody has any examples of their own of automated build integrations with one of these tools or others, I’d love to see them)

(Update) See the posts detailing how to integrate each of these into CruiseControl.NET here:
NDepend
vil
SourceMonitor

Yesterday I had the opportunity to hop on over to Oklahoma City for an Extreme Programming Workshop being given by Raymond Lewallen (thanks again Raymond).

In the midst of discussing the practice of Test Driven Development, Raymond asked me how many lines of unit test code I typically write per line of actual code. Not having used any line counters or code metrics software, I threw out a best guess number off the top of my head……oh, maybe 10 lines.

10 lines of unit testing code for each line of actual code!?

The more I thought about it, the more that number sounded pretty high to me. And Raymond mentioned that his average was typically around 17 to 10 or 1.7 unit test lines per actual line. So, I decided to do some investigating.

I fired up a line counting tool that Raymond had mentioned in his talk called countlines and pointed it to my most recent project. Here’s the verdict:

Actual lines: 3868
Unit Test lines: 4065
Ratio: 1.05 unit test lines per actual

While somewhat embarrassed that I was so grossly inaccurate, I wanted to share this especially with anybody else attending yesterday that might read my blog, as I find this number to be more reasonable and hopefully more in line with what others are experiencing – which I’d love to hear about!

What kind of ratios are you seeing on your projects?

Jeremy D. Miller has some really nice posts on getting started using Test Driven Development.

What has really turned me on to TDD (so to speak) is that it really forces you into a lot of good coding practices like Dependency Inversion and Low Coupling almost naturally as they are necessary for test isolation and avoidance of headbanging. Despite the fact that I’ve always known that I should write code with few dependencies, high cohesion, and low coupling, somehow when I wrote code before I stumbled on to TDD I never quite achieved these high ideals to the same degree. Now that I’ve been using test driven development techniques for awhile, I find these ideals easily incorporating themselves into my code without me thinking about it.

Jeremy’s posts show you techniques to achieve some of these practices and get you into the test driven groove.

In an earlier post, I discussed a technique I developed for manipulating the HttpContext within NUnitAsp tests. Well, one of the major impetus (impeti? impetuses? gaggle?) for this technique came from me banging my head against a wall trying to figure out how to create testing scenarios that would isolate the user interface from my business logic and/or data access.

I started out with the idea of creating a little MVC (Model-View-Controller) structure with the Model being the business domain object (or group of objects), the View being the aspx page, the code behind handling events from the View, and then creating a separate Controller that the code behind would delegate calls to as needed – primarily when it needed to interact with the business or data layer.

Something like this:

And then I thought to myself that man shall need a way to decouple the creation of the controller from the code behind so that it may be set by a test scenario. So I spake into the darkness to create a ControllerFactory, and it was created and it was good, and I saw that it was good.

But it wasn’t. Good that is.

This is where head-banging ensued (and not the Beavis & Butthead kind). When I tried to swap a mock controller into the factory using the static SetPageController method, it didn’t work. It didn’t throw an error, but the controller that the page used during the test was not the mock controller that I expected. Not cool. That’s when it hit me that the test was running in a different context from the actual page. I thought all was lost. All this for naught.

But, alas, all was not for (uh) naught. Not at all! As you already know, I discovered the workaround technique mentioned previously, so the day (and my head) was saved.

So, putting it all together using this technique and the MVC-like pattern, I was able to swap in and out my trusty mock controllers at will, allowing me to write focused tests on the user interface interactions like so:

  104 public static void ClickSearchButtonForASingleMatchingOrderID_ResultsPanelShouldBeVisible_Setup()

  105 {

  106   string orderID = "1";

  107   ArrayList collectionWithOneOrder = new ArrayList(1);

  108   collectionWithOneOrder.Add("One Order");

  109 

  110   DynamicMock mockController = new DynamicMock(typeof(OrdersSearchController));

  111   mockController.ExpectAndReturn("GetMatchingOrders", collectionWithOneOrder, orderID);

  112 

  113   ControllerFactory.SetPageController("OrdersSearch", (AbstractController) mockController.MockInstance);      

  114 }

  115 

  116 

  117 [Test]

  118   public virtual void ClickSearchButtonForASingleMatchingOrderID_ResultsPanelShouldBeVisible()

  119   {

  120   Browser.GetPage(this.BaseUrl + "Setup.aspx?op=OrdersSearch.ClickSearchButtonForASingleMatchingOrderID_ResultsPanelShouldBeVisible_Setup");

  121   Browser.GetPage(this.BaseUrl + "OrdersSearch.aspx");

  122   orderIDTextBox.Text = "1";

  123   searchButton.Click();

  124 

  125   AssertVisibility(resultsPanel, true);

  126   }

For a more detailed and complete code sample, get the source here.

Robert  has additional info related to setting up PostToSpaces that you mind find helpful:

http://www.geekswithblogs.net/jemimus/archive/2005/07/16/47180.aspx

Thanks Robert!