Coding and Dismantling Stuff

Don't thank me, it's what I do.

About the author

Russell is a .Net developer based in Lancashire in the UK.  His day job is as a C# developer for the UK's largest online white-goods retailer, DRL Limited.

His weekend job entails alternately demolishing and constructing various bits of his home, much to the distress of his fiance Kelly, 3-year-old daughter Amelie, and menagerie of pets.

TextBox

  1. Fix dodgy keywords Google is scraping from my blog
  2. Complete migration of NHaml from Google Code to GitHub
  3. ReTelnet Mock Telnet Server à la Jetty
  4. Learn to use Git
  5. Complete beta release FHEMDotNet
  6. Publish FHEMDotNet on Google Code
  7. Learn NancyFX library
  8. Pull RussPAll/NHaml into NHaml/NHaml
  9. Open Source Blackberry Twitter app
  10. Other stuff

Should Unit Tests Live in Production Projects?

As I'm doing more and more work with unit tests, I'm beginning to have some pretty unconventional ideas around the way we organise our test code and production code. My latest thoughts? I'm seriously considering putting all of my unit tests in my production projects and wrapping them in compiler conditionals so they're not included in release assemblies.

These aren't the worst suggestions I've heard - some people are even advocating including your unit tests in your production classes, just tagged at the bottom, I'm not sure I'm quite that far gone, but I'm getting there.

Where I Am Now - Separate Projects For Unit Tests

At the moment we're doing unit tests in what seems a fairly conventional fashion. Our solution tends to include projects that look something like:

Project.Application
Project.Domain
Project.WebUI
Tests/
  Project.Application.Tests 
  Project.Domain.Tests
  Project.WebUI,Tests 
  Project.WebUI,Specs

This has some benefits.  Your live assemblies are clean and ready to deploy, but also some people like this type of navigation - if you want to work with tests you go to one place, if you want to work with code you go somewhere else.  Here I think is the problem.

The Problems - Two Separate Worlds

My main beef with this way of working is the separation that it introduces between live code and test code. The closer I can get to my tests and my application being the same thing, the happier I am. A lot of new ideas I hear about in TDD, this coupling is getting closer and closer.  Two examples off the top of my head:

  1. The Builder Pattern - When I use this, I'm mirroring my production assembly depencies into my test assemblies.
  2. TDD As If You Meant It - This one says to write your production code in your tests, and move this code into production assemblies only when you need to.

Then there's the practical issue that in a larger solution, there's an overhead in doubling your number of projects.  It does take a measurable time to dig out unit test classes for your production classes if you're flitting between a large number of classes, and it does take a time for the builder to produce and link all these additional DLLs and debug files.

Finally, separate assemblies is what everyone else is doing - my non-conformist tendencies are probably somewhere in this story.

The Plan - Unit Tests In Production Projects

So my plan is to replace the above 7 projects with 4.  I'm going to keep my SpecFlow tests in a separate project, because they're full integration tests, but all of my other unit tests I'm going to put right alongside my production classes.  I'm thinking of a list of classes that looks something like:

Thermostat.cs
ThermostatTests.cs
ThermostatMode.cs
ThermostatModeTests.cs

Next, I'll wrap the contents of all my test classes inside a "#if DEBUG ... #endif" compiler conditonal, so that the code is only compiled in debug or mode.

A lot of you might be thinking what I initially thought - won't you get lost amongst all of these additional classes? Maybe, but I doubt it - one of the points Uncle Bob makes in one of his Clean Coders videos, is that the IDE gives us such great navigation tools, we shouldn't worry about getting lost in our code base.

What're The Benefits - A More Fluid Workflow

Caveat : I'm going to mention it again - this is all still theory in my head, I've not tested this yet!

I can see a major benefit to this approach being how much easier it is to fix up unit tests for all the myriad small changes I might need to make in my code while I'm refactoring.  I know on its own this may be a code smell, but certainly in legacy appliations I often find myself working on class A, then flitting to class B to fix a bug, which means I need to make changes to classes C, D and E.  In these cases I don't just want to run my tests and check I've not broken things, I often need to write new tests, and I want to do it there and then.

Then there's the build time - the quicker my app builds and the sooner I can get feedback from my tests, the easier the whole red-green-refactor cycle becomes. On larger projects, I think this will cut build time substantially.  We've got some applications that comprise 60 or 70 projects, and these take minutes to build every time we want to run our tests.  Conversely I've worked with some open-source projects that are only 3 or 4 projects, even though there're a similar amount of code and complexity, they build miraculously quickly.

What's All The Fuss?

When I've mentioned these ideas to colleagues at work, they've invariably recoiled in horror, as though I'd recommended we migrate all our code to VB.Net or we ditch Subversion and go back to Sourcesafe. There are a number of reasons behind this I think:

Clean Assemblies - Right now as coders we're very concerned with ideas of "clean code", maybe this practice isn't "clean"? I'm going to argue otherwise. In some cases, for example if you're working with an API, the ability to distribute your tests with your API is a really good thing. On the other hand remember that these tests are all wrapped in compiler conditionals, so they can be turned off for production builds if you want. There's also a sneaky feature in Visual Studio where you can have similar conditionals on your includes, so libraries such as Moq or NUnit are only included if your tests are enabled.

Risk - Moving code around like this is a heavy job.  If you've got 1000 classes, and you add into the mix 1000 test classes, all living right alongside your real classes, it's going to take some pretty heavy shifting to migrate from the old way of testing to this new pattern, or vice-versa.  In other words, once you've started along a particular road, the further you go the harder it is to go back. Why take the risk?

Convention - This one's partly my fault. On our team we've struggled historically with conventions, around 6 months ago I became a little teenie bit of a code nazi, and one of the laws that was laid down early on was that we had a one-to-one match with production assemblies and test assemblies.  Now that I'm saying otherwise, of course the guys think I've lost the plot.

I'm a Known Crackpot - Then there's the reputation I have at work for being something of a crackpot. Probably best I say no more on this one!

In Conclusion

Next thing to do is to put all of the above into practice. I'm not sure where I'll get the chance yet, but maybe I can find a nice squirelly little side-proect at work to test this out on, who knows. At the moment I'm around 75-80 percent sure this way of working would give us a net benefit, but time will tell.


Permalink | Comments (4)

Comments (4) -

David Toon United Kingdom

Monday, June 06, 2011 9:49 AM

David Toon

Just wondering if you would advocate running your tests against the "Release" build of your dll's?  As such if wrapped in the #if DEBUG then this wouldn't run against your "to-be" production build.  Maybe you could just run your Integration Tests against your production build, if obviously the "Debug" build passed the unit tests...  Maybe, it's not such a crackpot idea, after all!!!

The mind ponders...

Maybe it's a different problem altogether - it's about the size of the assemblies - and there relationships to one another...  I wonder how much assemblies should depend on one another!!!  I.e:
Domain - should be a separate solution, with just
Sln Project:
Project.Domain
Project.Domain.UnitTests

Therefore, breaking out your assemblies in to different versions - then deploying them to the GAC for your application to consume.

David Toon United Kingdom

Monday, June 06, 2011 9:50 AM

David Toon

Sorry read "tests" as Unit Tests!!!

russell United Kingdom

Sunday, June 12, 2011 4:24 PM

russell

Blimey, I spend half my time bitchin' about spam comments, but when I get a real comment it takes me a week to get to it!  Apologies dude.

When I started reading your comment, you've actually gotten the same conclusion as myself. Your integration tests still live in a separate assembly, so yes you can run these against your live assemblies. Although of course this only works if you trust your integration tests, and your production code's not riddled with "#if DEBUG" statements (shudder).

On the subject of a separate solution for your application's domain - I think there're legs in that idea, I'd be interested to read that fleshed out into a blog post?  (hint hint!)

Phil Yardley United Kingdom

Monday, June 13, 2011 9:50 AM

Phil Yardley


My 2p worth is that putting the tests into the same namespace as the production code would effectively double the size of the assemblies after compliation - there would be no way to seperate these pre-deployment. (why deploy 150MB of dll's when only 75MB is active code?) - this affects the memory footprint of your application on the server.

When it comes to automation - if the tests are in a seperate library, you can say "run all the tests in this library" - a one liner.  When putting them in with your production code - I'm sure (though I've not tried it), you would need to say "run tests in class1, class2,class3 etc) for each of your test classes. This results in a) more cranking of automation code. b) the risk that a test class is missed (added to the project but automation scripts not updated) - this increases the risk of bad code getting into production. And to echo Dave - if you use #debug - you can't test in Release mode?

I'll shut up not because I'm sure that's 5p worth and not 2p Smile


Add comment

  Country flag

biuquote
  • Comment
  • Preview
Loading