TDD Pattern: Tabular Tests

Originally published  5 Feb 2006

Today, I thought I'd blog about one of my most common testing patterns.  I call it table-like tests because I end up with a test method that calls a helper, parameterized test method.  The high level test method shows all the test cases in what looks kind of like a table.  To give proper credit to this technique, I believe I read about it on www.junit.org or one of the XP sites years ago, but I couldn't find the article with a quick Google search.  To demonstrate, suppose I were working on a Range class and was currently focused on the equals() method.  I'd end up with the following:

   private static final int MIN = 1;
   private static final int MAX = 7;

   private Range range = new Range(MIN, MAX);

   public void testEqualsAnotherRange() {
      testEqualsAnotherRange(MIN,     MAX, true);
      testEqualsAnotherRange(MIN - 1, MAX, false);
      testEqualsAnotherRange(MIN,     MAX + 1, false);   }

   private void testEqualsAnotherRange(int testMin,
         int testMax, boolean expectedEquals) {
      ...
   }

Now, I know there are other tests for the equals() method, but I'm focusing on the meat of the tests here.  I also want to emphasize that I don't start out specifying all the tests like this.  I actually write one of the test cases, get that test to pass by minimally implementing the Range class, and then, as I start to write the next test case, I see the pattern and will extract the method and parameterize it for the next test case.
Time to make some points:

1. Test cases for the equals() method are grouped together.  If I want to know how the equals() method works for non-exceptional cases, all the test cases are right there. When I write code, I focus on a method for a particular production class.  That is, I'm trying to build out all the functionality for that method before moving on to the next method (or class).  I find it helpful to have all the test cases grouped together.  Parameterizing tests keeps the test cases as close together as possible.
2. Whenever I detect duplication, I strive to remove it.  This is a way to do so among the test cases.
3. The mapping between test methods and production class methods is nearly one to one.  There's one main testEquals() method for one equals() method.  Of course, I'll have test methods for exceptional cases, but I don't have much more test methods than production methods.  If I kept each test case in its own method, I'd have a harder time removing duplication and I would have much more test code than production code.

I should also point out that many times, I don't use the same fixture.  In the example above, I did, but many times I don't.  The parameters passed in to the test helper method could be used to setup the production object.  Sometimes, the parameters are used to help mock a nested object.   Take, for example, the typical web site Order class.  Let's suppose I were focused on the getTotal() method:

   private Order order = new Order();

   public void testGetTotal() {
      assertEquals(0.00, order.getTotal(), 0.0001);
      testGetTotal(1, 3.00, 3.00);
      testGetTotal(3, 5.00, 18.00);
      // etc.
   }

   private void testGetTotal(int lineItemQuantity,
         double lineItemPrice, double expectedTotal) {
      // create a line item
      // add line item to order
      // verify total
   }


Now, I should point out some of the concerns of using this technique:

1. It doesn't follow the JUnit convention of working with the same fixture per TestCase.  Many times, my "fixture" is just an empty production object that will be setup in individual test methods.  Behaviour Driven Development (BDD) is getting press these days and its focus is on fixtures called Contexts.  Caveat: my knowledge of BDD is from reading this one article by Dave Astels, and I haven't experimented with it.  It seems to be good for showing behavior in a given context, whereas my technique is method focused.  Maybe I'll give it a shot in the future.
2. Because there isn't a real fixture, the test methods are more static than object-oriented.  That is, I might not be working with instance variables of the TestCase.  I don't find this to be an issue though.  I thought I'd point it out for OO purists.
3. Gerard Meszaros, who is writing a good book on test automation patterns, pointed out that if one of my earlier tests fails, then later tests won't be run.  He also pointed out that it is harder to determine how many total test cases there are since some are embedded.  These are valid points.  To counter, I mentioned that I am following test-driven development, so I am building the tests as I go, and I usually don't run into the situation where an earlier test fails, although it does happen.  I also never run into the situation where a bunch of tests are all of the sudden failing.  As for the number of test cases, I've never had a need to know.  But his points are definitely valid.
4. Tests aren't isolated.  You don't start fresh each time.  Thus, earlier tests could have unwanted side effects.  You have to be aware of this.  I either want side effects (as in the Order example where I'm building up an Order) or I simply start with an empty production object and build it up in the helper test method.