Spring Configuration Per Environment

Originally published 30 Jan 2008

It's common that configuration changes between environments (e.g., development, test, production).  For example, you'd definitely want a different DataSource URL depending on what environment you're running in.  Do you really want to hit the production database while developing?  Spring doesn't seem to support this per environment configuration out of the box (at least I've never been able to find it).  There's been a request for this feature for some time now (SPR-1876), but it's still open.

The reality is that properties files are the simplest and easiest way to configure an application, particularly for non-developers.  What I'd like to see is a PropertyPlaceholderConfigurer that can configure an application per environment.  This is what SPR-1876 is all about.  However, it suggests there be a file per environment.  As an alternative, I suggest a single properties file.  A single file containing all the properties would be the simplest to maintain.  Think of it.  I've got a new property.  Do I want to add it to three files or one?  If I want to see all the possible values of a property, do I want to open and hunt through three files or one?

Grails supports the idea of grouping all the properties together, but it uses multiple configuration files (Config.groovy and DataSource.groovy) depending on the property.

Okay, so here's what I propose the properties file look like:

# this is the default value for all environments
my.prop=defaultValue
# but in the production environment, it is different
[production].my.prop=productionValue
 
# here's another property with no default
# (adapted example from the Grails user guide)
[development].dataSource.url=jdbc:hsqldb:mem:devDB
[test].dataSource.url=jdbc:hsqldb:mem:testDb
[production].dataSource.url=jdbc:hsqldb:file:prodDb;shutdown=true
 

The context definition simply uses the property name without the environment and the custom per environment PropertyPlaceholderConfigurer provides the appropriate value:

<bean id="dataSource"
      class="org.springframework.jdbc.datasource...">
  <property name="url" value="${dataSource.url}" />
</bean>

So what are the downsides to this approach?  I suppose you can argue that a single properties file could be cluttered with details you don't care about.  For example, a production administrator may not care to see all the development and test values.

The other downside I can think of is forgetting to override a property.  That is, nothing reminds you that you have to override the DataSource URL for production.  It's content with using a default value.  The separate file approach would handle this because the property would be undefined.

So if I were reading this I'd say, "Spring is open source dude.  And those Spring guys are busy.  Go implement it and attach it to SPR-1876."  And if I get some time...

Law of Demeter and Unit Test Setup

Originally published 30 Nov 2007

Have you ever seen or even developed a long, complex setup method in JUnit?  Maybe it was because the production code was violating the Law of Demeter.  Here's another Robert Martin Craftsman style blog.

Apprentice:  Journeyman, I'm having trouble writing the fixtures for my JUnit tests.  They take too long.  I spend about 10 times longer writing tests than the production code.
Journeyman:  Really?  Unit test expert Gerard Meszaros says that tests should take only 10% to 20% of development time.
Apprentice:  Well, I'm experiencing the exact opposite.
Journeyman:  Let me take a look.
Apprentice:  Sure.
Author Note:  This is a trivial example for this blog entry.  I've seen some really obscure test setup methods.  The example is based on the sequence diagram chapter in Martin Fowler's UML Distilled book.

 public class OrderTest {
 
     private Order order;
 
     @Before
     public void setUp() throws Exception {
         Product product1 = new Product(11.00);
         OrderLine line1 = new OrderLine(2, product1);
         
         Product product2 = new Product(22.00);
         OrderLine line2 = new OrderLine(3, product2);
 
         OrderLine[] lines = new OrderLine[] {line1, line2};
         order = new Order(Arrays.asList(lines));
     }
     
     
     @Test
     public void priceShouldBe88() {
         assertEquals(88.00, order.getPrice());
     }
 }
  

Journeyman:  Okay, can I see the Order.getPrice() method?

     public double getPrice() {
         double price = 0.00;
         
         for (OrderLine orderLine : orderLines) {
             Product product = orderLine.getProduct();
             double productPrice = product.getPrice();
             int quantity = orderLine.getQuantity();
             price += productPrice * quantity;
         }
         
         return price;
     }
 

Journeyman:  Ah, I see the issue.  Have you ever heard of the Law of Demeter (LoD)?
Apprentice:  Huh?
Journeyman:  The LoD has a few different names and related principles, but basically it means that an object should only deal with and only with its collaborators.
Apprentice:  Huh?
Journeyman: You see how Order iterates through its OrderLine objects?
Apprentice:  Yeah.
Journeyman:  Well, that's fine.  But then you grab the Product and then grab its price.  You've violated the LoD.  Order should only know about OrderLines.  You've got more of a procedural design here, where you reach down into all the objects in the graph, grabbing all the data you need, and then do your thing.  A more object oriented design would be to distribute the work.  Do a little bit of work and delegate the rest to collaborating objects.
Apprentice:  Oh, I think I'm beginning to see.
Journeyman:  You should go read up on Craig Larman's GRASP principles.  He calls the LoD Don't Talk to Strangers.
Apprentice:  Okay, will do.
Journeyman:  Have you been following test driven development (TDD)?
Apprentice:  Uh..., well..., not really.  But I do write the tests afterwards.
Journeyman:  Complex test fixtures jump right out at you when you're doing TDD.  You realize something is wrong right away.  In fact, many times your testing style changes into more of an interaction based style.  But I'm getting ahead of myself.  Let's refactor this together.
Apprentice:  Okay, cool, pair programming.
Journeyman:  Right.  Remember, Order should only deal with OrderLines.
Now, the setUp() method looks like:

     @Before
     public void setUp() throws Exception {
         OrderLine line1 = stubOrderLineGetPriceToReturn(22.00);
         OrderLine line2 = stubOrderLineGetPriceToReturn(66.00);
 
         OrderLine[] lines = new OrderLine[] {line1, line2};
         order = new Order(Arrays.asList(lines));
     }
 

And the Order.getPrice() method looks like:

     public double getPrice() {
         double price = 0.00;
         
         for (OrderLine orderLine : orderLines) {
             price += orderLine.getPrice();
         }
         
         return price;
     }
 

Apprentice:  Okay, now I really see.  Order and its test are simpler, but don't we have the same amount of work anyway?  We had to write OrderLine.getPrice() and test that.
Journeyman:  Yes.  The work is distributed.  But I prefer lots of simpler objects with simpler test specifications than more complex objects and tests.
Apprentice:  What do you mean, more objects?  We didn't create any new objects.
Journeyman:  I meant generally speaking, not necessarily in this case.  Like I said, do a little work and pass the buck.  That might mean creating Data Clumps, aggregate objects for collections, objects that represent abstract data types, and so on, but again, I'm getting ahead of myself.
Apprentice:  Should I always follow the LoD?
Journeyman:  Well, there are of course consequences.  Classes tend to have larger APIs because sometimes they need to wrap the functionality of collaborating objects.  And you need to step through more objects to understand an algorithm as a whole because it's distributed.  But most of the time, I follow LoD because dependencies are reduced.
Apprentice:  All right, I'll give it a shot.  Thanks.

Parallel JUnit Ant Task

Originally published 16 Oct 2007

Over time, how do you maintain a maximum 10 minute build? This is an important agile practice for continuous integration.

It's inevitable. As more and more features are added to an application, the code base grows. The build has more to do: more compiling, more tests and metrics to run, more reports to generate. We've done numerous things to keep the build time down. In this entry, I'd like to describe one of them: running standard JUnit tests concurrently.

A colleague of mine came up with this idea. He first did a preliminary search to see if anything already existed.  Pretty much everything he found was intrusive, e.g., requiring extending a specialized TestCase.  He then attempted to combine Ant's Parallel and JUnit tasks. That is, he could simply nest N number of JUnit tasks inside a Parallel task. The problem here was coming up with a good way to divide the tests up into equal parts beforehand. My colleague quickly scrapped this idea in favor of a custom Ant task that wrapped the JUnit task.

The thought was to leverage the JUnit task as much as possible, but fronting it with the ability to run tests in parallel. The conceptual design looks like this:

There are N number of TestExecutors. A TestExecutor runs in a single JVM. Thus, there are N JVMs. Say we have a dedicated, 4 processor integration machine running builds. We may choose to configure our custom task with 5 TestExecutors (we'll add one assuming we're not 100% compute bound). Note: if you're familiar with the JUnit task, you may be wondering how its fork/forkmode attributes fit in. The answer is that they are eliminated in favor of this jvm-count attribute.

There's a single TestProvider. His job is to gather all the tests and hand them out one at a time to whichever TestExecutor is ready. A simple protocol exists to let a TestExecutor tell the TestProvider that he's done and ready for another test.  This should maximize parallelism.

The biggest negative with this approach is that since all tests run in a fixed number of JVMs, there's some loss of isolation here. Class level/global state set from a previous test could affect later tests. We're willing to trade this for increased performance.

The other possible downside is the use of a non-standard Ant task.

Using this approach we've successfully reduced our build time to usable levels.

The Continuous Integration Build Game

Originally published 14 Sep 2007

I was reading Alistair Cockburn's Agile Software Development book the other night. In it, he described a game developed by Darin Cummins to reinforce good development practices. The game is described here. This inspired me to create a fun game for continuous integration builds.

As I described in a previous blog entry, I've had problems in the past with people breaking the build. To help, I've used techniques like the rotating Build Nazi and the put a dollar in the broken build jar, but these are all negative focused. How about something that rewards developers that don't break the build? How about rewarding developers for following the best practice of breaking their work into smaller chunks and checking in early and often?

So I'm thinking of a game where a developer gets, say 1 point for getting his name on a successful build. The number of files checked in is ignored. We want to discourage big check-ins, so you get more points for smaller grained check-ins since your name will show up on more successful builds. And on the other side, you get points taken away when you break the build. Now, we want to keep things simple, so we could probably stop right here, but I'm thinking that some failures are worse than others. So maybe we have something like:

Description	Reward Points
Check-in and build passes	1
One or more unit tests failed	-10
Compiler error (come on now)	-20
Big check-in caused build to remain in a broken state for hours	-40

It's a game, so we need something for the winner. Bragging rights is too lame, so maybe lunch on the team, or some kind of trophy kept on the winner's desk to remind everybody of his champion status.

Now, there are some negatives. Perhaps not everybody would want to play. Particularly, notorious build breakers wouldn't want everybody (specifically management) to see their poor results. In that case, I suppose we could only publicly display the leaders, or top half, but that wouldn't be as fun.

People could easily cheat too. Maybe, write a cron job that every hour checks out a file, changes a comment and checks back in. We'd have to look out for that kind of thing.

What about the analysis time required to keep score? I could easily see how a post processing ant task could be developed to update the points for developers on a successful build. But for a failure, I think you'd need human analysis. That's a negative because it requires time, so the job could be rotated. On the plus side, what I've noticed is that analyzing why the build failed brings awareness to issues. Issues like some people needing training, or a test requiring change because it's non-deterministic, or a hole in the pre-check-in process used to ensure a successful build.

To keep the game fresh and the developers motivated, we'd have to reset the points periodically. Iteration boundaries seem appropriate here.

Well, maybe I'll give it a shot and see what happens...

The Difference Between a Property, Field, Attribute

Originally published 3 Aug 2007

I’ve been asked a similar question at least three times this year.  It goes something like, “What’s the difference between a field and a property?”  Or, “Is there a difference between an attribute and a property?”  Given that many inexperienced developers automatically generate getters and setters for all their fields, I can understand the confusion.

I always give them the example of a circle.  Here’s some Groovy code:

class Circle {
   def radius
   
   def getDiameter() {
      2 * radius
   }
   
   getArea() {
      Math.PI * Math.pow(radius, 2)
   }
   
   def getCircumference() {
      2 * radius * Math.PI
   }
}

radius is a field, also known as an instance variable, also known as a member variable in C++.  It’s an implementation detail of the Circle, i.e., it’s a private variable that gets stored as part of the Circle object.

Properties are more public aspects of an object.  In Java, following JavaBeans conventions for getters and setters allows you to expose properties.  Those properties don’t have to be backed up by fields, but in many cases are.

Coming back to the Groovy Circle, what are its properties?  We’ll you get radius for free as a read-write property.  But because we’ve got some getters that compute other values based on radius, we also have diameter, area, and circumference properties.  These happen to be read-only properties since we didn’t provide setters, but we could have (which would update the radius field accordingly).  You can dump all of a Circle object’s properties in Groovy like this:

println myCircle.properties

The point is Circle could be implemented in different ways: by using a diameter field instead of radius, for instance, but it would have the same properties.

Now, let’s talk about attributes.  I typically think of UML when I hear attributes.  Here’s UML for Circle:

Notice that I’ve distinguished the derived diameter, area, and circumference attributes with a “/”.  This is standard UML.  radius is a regular, non-derived attribute.  We can distinguish fields from other computed properties like this.  However, because we’re modeling, and thinking in a higher level of abstraction, I like to think of attributes as more synonymous with properties.  I’d rather not make the assumption that every attribute is implemented with a field.

A similar discussion of these terms can be found here.

How to Refactor Many Arguments

Originally published 25 May 2007

The following conversation is based on a true story.  The roles are taken from Robert Martin’s Craftsman series.

Apprentice:  Hey, can I ask you something?
Journeyman:  Sure.
Apprentice:  I was looking at a big constructor on one of our domain objects and I saw some groupings and was wondering whether it would be good to create objects for those groupings.  Is there some kind of pattern for that?
Journeyman:  Well first, how many parameters are there?
Apprentice:  72.
Journeyman:  72!?  Wow.  Talk about a code smell .  PMD's ExcessiveParameterList rule will flip out on this one.  I guess the author mapped those parameters straight from the out-of-our-control XML schema, whose document instances get unmarshalled in order to create the object.  We don’t have to follow suit and create such a flat domain model that matches the schema.
Apprentice:  What do you mean?
Journeyman:  Well, what I really mean is that I'd much prefer distributing the logic among some richer objects.  That is, create more little objects with little methods that do a little of the work.  Not one big object that does it all.  There would be an impedance mismatch between XML and the domain model, but that’s typical because of the technology differences.  I suppose the only benefit of the big constructor approach is the simpler mapping... maybe, but the negatives are much too great.
Apprentice:  What are the negatives?
Journeyman:  Well, for one, trying to list all 72 parameters in the correct order would be a pain.  And trying to understand the object as a whole with all those fields would be difficult.  Here [grabbing his Refactoring book].  You were initially asking about the “Introduce Parameter Object” refactoring.  Read that refactoring.  It can probably give you better details.
Apprentice:  Martin Fowler.  Does he know what he’s talking about?
Journeyman:  Oh yeah.  He’s one of the Masters.  Let’s take a look at the class.  [Brings up the class in the IDE]

public ImportantDomainObject(
   ...
   int x, int y,
   ...
   double min, double max,
   ...)

Well, I can see a few data clumps already.  You see the x and y?
Apprentice:  Yeah.
Journeyman:  That’s probably a Point.  Also, that min and max, that looks like a Range.  What I would do is search for data clumps like these and create objects for them.  Then, see where they’re used in ImportantDomainObject and start moving behavior into the new objects.  Distribute the logic.  We can create a richer domain model this way.  You’ll probably find clumps of these clumps and can follow the same process.
Apprentice:  Okay.  This sounds great.  Thanks.  I’m on it.
Journeyman:  Thank you for recognizing this and taking the initiative to improve the code base.

DBC Precondition/Postcondition Subclass Rules

Originally published 12 Apr 2007

I was recently in a discussion regarding the precondition and postcondition covariance and contravariance rules of design by contract (DBC) in an inheritance/implementation hierarchy. In order to obey the Liskov substitution principle (LSP) , a subclass or interface implementation can only meet or:

• Weaken the preconditions of the base class, not strengthen them (contravariance).
• Strengthen the postconditions of the base class, not weaken them (covariance).

There was some confusion as to why this was the case. So I came up with an analogy to explain. If you think in terms of the client requesting services of an object specifying the preconditions and postconditions, it all makes sense.

So the analogy? Have you ever had to request a help desk ticket? Maybe you need administrative rights or something. And you hope that a specific someone is assigned to that ticket because she's really good and can get the job done quickly and accurately. Let's use that as an example.

The general IT support group contract is:

• Precondition: Submit a help desk ticket filling in all the forms on the web site.
• Postcondition: We'll get back to you within two hours.

Now, suppose Suzy Support works in the IT support group. She's really good and efficient. You can think of her as a subclass of the IT support group base class. In the past, whenever I've had a problem and I just happened, purely by chance, to pass her in the hallway and mention it to her, she could solve it in minutes. From my perspective, as a client of the IT support group, that's acceptable.

Suzy's contract is:

• Precondition: Just tell me the problem. You don't have to fill out a help desk ticket for me. This is weaker than the IT support group's precondition because I have less work to do.
• Postcondition: I'll solve it in minutes. This is a stronger postcondition. I get better response time.

Make sense?

By the way, for all the IT support group managers out there, this is just an analogy. I know bypassing the help desk ticketing system messes up metrics.

Groovy or JRuby?

Originally published 24 Mar 2007

Which dynamic programming language should I introduce to a group of relatively young, inexperienced Java developers? The choice is between Groovy and JRuby.

Ruby and particularly Ruby on Rails has been hot for awhile now. Ruby in the enterprise is becoming more accepted. As Martin Fowler pointed out nearly a year ago in this blog entry , some top minds in our field including Bruce Tate, Justin Gehtland, and Stuart Halloway have moved Beyond Java . Even one of my old mentors, Mike Gaffney , has repeatedly told me to bail on Java. But I agree with Scott Davis as he points out in this podcast , Java is not dead.

The growing acceptance of Ruby is a strong reason to consider JRuby. Being able to run Ruby on Rails apps on the JVM is very appealing. But remember the context of the decision here. We're talking about a group of developers whose only development language has been Java, and whose projects for the foreseeable future will run on the Java platform. Groovy wins in this context.

There are lots of reasons to get excited about Groovy. It too runs on the ever-improving JVM. In fact, it was the first scripting language officially approved by Sun to run on the JVM. It can be compiled to byte code or run in script form. The interoperability is fantastic. Developers can still use Java, its tools and familiar libraries such as Spring and Hibernate , where appropriate, and mix in Groovy, where that is more appropriate.

But what really makes this decision easy is the similarity in language syntax. You can take a snippet of Java code, drop it into a Groovy file, and it will nearly work. Then, you can take that snippet and start making use of Groovy language improvements such as closures to make the code more readable and intention revealing .

So the decision is easy here. Take the seamless step forward and introduce Groovy.

Bad PMD Rules

Originally published 9 Mar 2007

PMD is an excellent tool for finding potential bugs and improving code quality, but it can generate a lot of false positives. Here's my list of the top 10 rules I turn off immediately, in alphabetical order, with a short comment explaining why. Descriptions for the rules can be found here.

1. AtLeastOneConstructor: Why? Code is smaller without an empty, no-args contructor.
2. AvoidInstantiatingObjectsInLoops: This one just seems to generate too many false positives. In addition, for short-lived objects, garbage collection is essentially free.
3. CallSuperInConstructor: super() is called implicitly and requiring it just adds more lines of code.
4. JUnitAssertionsShouldIncludeMessage: Most of the methods in Assert already generate good messages. I do include a message for those that don't (assertTrue(), assertFalse(), fail()).
5. LocalVariableCouldBeFinal: final for variables is usually overkill and it adds line length.
6. LongVariable: Clarity rules!
7. MethodArgumentCouldBeFinal: Same as LocalVariableCouldBeFinal.
8. OnlyOneReturn: I think it's clearer to exit early. It reduces the amount of things to think about below the exit.
9. PositionLiteralsFirstInComparisons: This really isn't that bad, but myString.equals("x") just reads better than "x".equals(myString).
10. ShortVariable: There are just too many cases where it's acceptable to have short variables in small methods.

 These two just missed the cut:

• ShortMethodName: I try to make my names expressive. I've never seen this rule fire.
• SignatureDeclareThrowsException: This is an okay rule, but for JUnit tests, who cares?

The todo Test Category

Originally published 26 Jan 2007

Recently, Elliotte Rusty Harold blogged about committing test cases for bugs to the build cycle before fixing them right away. He enumerated several good reasons for doing this such as the fix being difficult and time consuming. His point was that breaking the build (where breaking the build includes passing all the unit tests) is okay in some cases.

I disagree with this stance, but I think Harold has a really good idea there, which I'll get to in a moment. I think breaking the build does include passing all the unit tests and should be part of the continuous integration process. I've always liked the idea of the tests serving as a safety net when I add new functionality or refactor existing code. The tests not only specify the desired behavior, but act as a regression suite.

So what's the idea? I first learned of the concept of test categorization from Cedric Beust's TestNG framework and the writings of disco king Andrew Glover . Essentially, the concept is to break tests into groups such as unit, component, and system and run them at different intervals. Unit tests are run many times a day as code is developed, component tests less frequently, and system tests the least frequently of all. You can categorize tests in several different ways such as by file naming conventions, placing them in different directories, or using Java annotations.

Harold brings up a new category - the todo category. (I was originally thinking the fixme category, but I thought todo could serve more purposes.) The todo category contains all tests that serve as a reminder of work to do. They obviously fail or there would be nothing to do. When it's finally time, the test moves out of the todo category and into the unit, component, or system category as appropriate. Although you could have a todo category for each of the original, regression categories (todoUnit, todoComponent, and todoSystem), I think that would be overkill. Notice that I slipped the word regression into "original categories" above. I'm advocating that the original categories should pass at all times - they should break the build if they fail (well, at a minimum the unit category). Once tests get into the regression categories, they never go back to todo.

The obvious downside of this approach and of test categorization in general is the added complexity. It's more work to maintain categories and in this case, move tests into different categories than a simple one category fits all approach. Also, there's more work (at least initially) to setup the running of those categories. As always, I recommend weighing the options and deciding what's best for your particular situation. See Glover's writings for more benefits of test categorization.