Python Rocks! and other rants - Agile Development 2004/5

Weblog of Kent S Johnson

Uncle Bob says, "Go well, not fast"

Unit testing a complex procedure

I am working on a unit test for a complicated, multi-step procedure. Conceptually it is something like this:

Ideally I would like to write tests for each step:

The problem is that the setup for each step is complex. The best way to set up to test step3() is to do step1() and step2(). So I have settled for a single test method that has the same structure as complicatedStuff():

This smells. It is a clear violation of Don't Repeat Yourself - the structure of complicatedStuff() is duplicated. As a result it is fragile. If the structure of complicatedStuff() changes, test_complicatedStuff() has to change the same way. On the other hand, it works, which is worth a lot!

Database unit testing is HARD

I am working on a project that makes some brain-twisting changes to a database. It is the first database work I have done in a while, the changes are a bit tricky, and the consequences of failure grim (breaking large production databases in use by many thousands of customers daily), so I am writing unit tests for everything using DbUnit.

DbUnit has one feature that I really like - the setUp() method of a test case can initialize the database to a known state. This feature alone is enough to adopt DbUnit.

What really stands out is how slowly I am making progress. There are many reasons for this - I am learning the problem domain as I go along, the problem has a number of wrinkles to it. But part it is that writing the unit tests is just plain hard.

Often unit testing is pretty simple - pass a few parameters to a function, check the result. Repeat for a few different sets of parameters. Sometimes there is a structure to be set up or checked. It's easy to work in small bites.

For this project, each test case requires the database to be set up. With DbUnit, this means creating an XML file that reflects the desired state of the database tables. These files are hard to read and hard to create when the table has foreign keys to another table. In my case, one of the tables represents a tree structure so it is essentially a list of parent-child relationships.

So first I have to figure out what will make a good test case. Then I create the XML file, either by hand editing or by somehow getting the database into the desired state and dumping it to XML. Finally I can write the actual test. This usually involves writing some queries to figure out if the database is in the correct state.

Then I can actually write the code to make the test pass. No wonder it is going slowly!

By the way I am writing the project in Jython. DbUnit works well with Jython because you don't have to subclass a DbUnit test case class - you can use DbUnit using independent objects and static assertions. I am writing the test cases with Python's unittest module and calling DbUnit as a library.

When to design, when to code?

Thinking about when to design and when to write code leads to an illuminating distinction between different development styles.

A (sadly) popular style of development is code-and-fix programming. In this style, the goal is to do the minimum required to get something that appears to work. At its worst this is coding without design, at any rate without thinking seriously about design or architecture or long-term viability. It leads to unreadable, unmaintainable code, cascading defects and many other problems.

Big Design Up Front (BDUF) is a response to this style. BDUF attempts to figure out the solution in the abstract before starting to implement it. This is design without coding. This approach has many problems as well. It is hard to figure out a good design without the hands-on knowledge you get from writing code. It is brittle and unresponsive to changing requirements. It takes considerable effort to create and maintain design documents.

The agile approach is a middle way. It disdains both sloppy hacking and excessive design. Agile programmers think about what might work and try it. They refactor when they have a better idea or when the requirements change. They keep growing the design.

Agile development is sometimes feared because without up-front design it must be "just hacking". If you aren't designing up front, with formal process and documents, you must not be designing at all. This fear is born from the reaction to code-and-fix programming and thinking that code-and-fix is the only alternative to BDUF. This is a false dichotomy and a false perception of agile development.

Design and coding work best when taken together. If you try to think your way through the design without writing code you lack the on-the-ground knowledge you get from coding. If you just start writing code with the sole goal of getting something that seems to work, you are hacking in the worst sense of the word. But if you consider each addition to the code carefully and keep the code clean at all times you will end up with a thing of beauty - well-designed code that is superbly suited to the task at hand.

Note: Ned Batchelder's blog about the similarity between diamond cutting and the decisions that must be made while designing a software system was the spark that led me to this idea.