I’ve been doing various degrees of Test Driven Development for several years and am still a strong advocate, but I’m definitely in one of those stages now where I am rethinking my approach.
The last time I found myself in this position was several years ago during my pre-mocking days when my test suite was taking an hour to run, the majority of my test failures were data-related ghosts, and the sql-laden setup and teardown sections of tests were painful to create and even more painful to maintain.
Now I find myself facing similar types of friction even though the causes may be different. Tests require too much effort to write due to mocking requirements, especially in areas of legacy code that need to be refactored to use dependency injection. Test failures are too often false positives caused by simple refactorings due to the tight coupling caused by interaction-based testing. The overall number of tests is difficult to manage from the perspective of documenting system behavior or being able to quickly discover whether a test already exists for a particular piece of functionality that is being modified.
As a result, our department has clearly stalled in its effort to incorporate TDD into our everyday development process and now I’m trying to figure out how to rectify that.
My current thinking is that the best way to make TDD a viable and sustainable option is to selectively use it only when it provides a net gain in value.
I know that this is somewhat of a blasphemous thought because TDD\BDD is more about design than automated testing, which means that it is supposed to be baked into the development process and therefore not optional (or so I thought).
Although I have definitely experienced moments where TDD has improved both the usability and elegance of my design by making me start from the perspective of the API consumer and incrementally add features in the simplest possible way that works, I have also seen plenty of cases where it provides little or no benefit in terms of design. For example, I often seem to write code that must fit into an existing design, follows a well known pattern, or only slightly modifies an existing method.
The same holds true for refactoring. While there have definitely been times where TDD has improved the quality of my code by providing a safety net that allows me to catch more bugs and refactor more freely, there have also been times when I’ve gained little value in this respect due to the simplicity of the code. In fact, lately I’ve noticed that there have actually been times when I have been less likely to refactor a section of code due to TDD because it had a large number of interaction-based tests around it that would all have to be changed to accommodate the implementation changes.
Finally, let us not forget that any and all code is intrinsically a liability. Since test code tends to grow at a much faster rate than normal code, the long term maintenance cost of your test suite becomes even more of an important factor to consider.
Ultimately, it seems as though some criteria for making a simple cost-benefit analysis are in order to help decide when it make sense to use TDD.
So far I’ve come up with the following:
When to write a test:
- The design is unclear.
- You’re implementing an important or complicated piece of business logic.
- You’re about to implement a hack or sub-optimal solution because you’re afraid of breaking something in an existing piece of messy, complicated code.
It also occurred to me that just because you write a test doesn’t mean that you have to commit it. There are several design artifacts (whiteboard, napkins, etc) that are extremely helpful despite being disposable. Occasionally writing throw-away tests would certainly allow me to gain the design benefit without incurring the cost of maintaining a test that has little long term value.
With that in mind, the following scenarios seem like reasonable examples of when it makes sense to take the extra step of committing a test:
When to commit a test:
- The code is important to the business and heads will roll if someone unintentionally breaks it.
- The code is complicated and the next poor schmuck who touches it to make an enhancement has a good chance of unintentionally breaking it.
Other ways to improve the cost-benefit ratio of testing:
- Use SRS to isolate important business logic – By favoring small classes and composition and focusing your testing efforts at this level rather than a higher level where they all meet, you will drastically reduce the setup cost of mocking out all the dependencies that probably aren’t relevant to your test anyway.
- Favor state-based testing over interaction-based testing – I’ve run into a few scenarios where interaction-based testing has been a blessing, but it seems like more often than not it just leads to inappropriate coupling with the implementation details that results in brittle test code. Use it sparingly.
- Make test suite spring cleaning a post release ritual – Nothing succumbs to entropy like code. Test code is even more critical to keep clear, concise, and organized, because this has the potential to represent the specifications of how your system works. If something is no longer important delete it. If the test name isn’t clear or the folder structure isn’t optimal for finding features, then invest the effort to change it. Otherwise, the broken window theory will take affect faster than it takes Visual Studio to load.
Am I way off base here? Does anyone else have additional criteria for deciding when it makes sense to remove yourself from TDD mode?
Popularity: 2% [?]
