Project Management

Do I Really Have to Test Everything? (part 3)

From the Sustainable Test-Driven Development Blog
Test-driven development is a very powerful technique for analyzing, designing, and testing quality software. However, if done incorrectly, TDD can incur massive maintenance costs as the test suite grows large. This is such a common problem that it has led some to conclude that TDD is not sustainable over the long haul. This does not have to be true. It's all about what you think TDD is, and how you do it. This blog is all about the issues that arise when TDD is done poorly—and how to avoid them.

About this Blog


Recent Posts

Do I Really Have to Test Everything? (part 3)

Do I Really Have to Test Everything? (part 2)

Do I Really Have to Test Everything?

TDD Tests as “Karen”s


Categories: TDD

This is part 3 of a three-part posting.  If you have not read part 1, I strongly suggest you start there:

Answer #3:
TDD is not about testing in the first place.  It never was.

The idea of driving development from tests can be a bit confusing to some.  "How can you test something you haven't created yet?  The test will fail, obviously.  What's the point?"

That a reasonable question too.  The fact is that TDD is not really a "testing" activity at all.  It produces enormously beneficial tests, but they are a side effect.  TDD is all about creating a specification - we simply use tests to do it.  It's not at all surprising that we would do that first, is it?  Realizing that TDD is actually a specifying activity also changes almost everything about the way we do it, and for the better. I could talk about that at length. But to the point I'm addressing here...

When you think this way, then the original question on the table becomes:

Do I Really Have to Specify Everything?

How about getters and setters?  They are so very trivial.  What about an object with no constructor, or only a parameter-less constructor, do I have to specify what "new" does?  It can seem like a silly idea and one that may well be a waste of time.

Well, first of all you don't have to do anything.  This is not "Test-Dictated Development", you are in charge of how you use your time.  However, you may find great value in specifying everything. I'm going to argue here that you will, and that you'll want to do it with automated tests.

When you create software you do so from an understanding you hold in your head.  That knowledge has value but it may leave your head over time.  Or your head may leave (you get another job, get promoted, win the lottery and retire, etc...).  When this happens, this represents lost enterprise knowledge, a significant problem across our industry. Many organizations have large, complex legacy systems that are mission-critical but the expertise that created them is long gone.  Updating them is dangerous, slow, and error-prone.

As an example: If you work on something to completion, then move on to other work, then perhaps months or even years later are required to work on it again you may well have forgotten a lot of what you knew when you made it. How will you re-gather the knowledge you once had in order to become qualified to do this work now?  Will you read the specification you left behind?

Imagine such a thing done in the traditional way: a document of some kind.  Its value was that it informed the initial development effort.  But can you use it later to re-educate yourself?  It is still accurate?  How do you know that someone has not changed the way the system works and failed to update the specifying document? That's not supposed to happen but it does all the time.  People get busy, run out of time, or don't even know that the document exists or how to find it.

The main problem is that you have no way of knowing.  You can't trust it. Its value is gone.  You may as well throw it away.

Now imagine the specification is a suite of tests.  You can execute them, and if they still run and pass you know that they are still accurate to the system as it stands today.  A big part of the way we write tests in TDD is to make them highly readable by humans, and so now you can read them and know the information you're getting is accurate.  But... what if they don't execute, or don't pass?  Then the specification has lost its value anyway.  What do we do about that?

You adopt the following strict policy: any change to the system must begin by changing the test that specifies that behavior.  The test will fail, but then the change is made to get it passing again. If you always do that, then the suite of tests will always represent a complete and accurate view of the way it worked, works, and has changed over time.  Forever.

Of course, that is only possible if everything has a test to change in the first place, right?  That's how you know the suite is always an accurate specification of the system.

So far so good. But... is it complete?

How do you know that someone has not added totally new behavior to the system and failed to add a test to the specification? It may be accurate as far as it goes, but it may be missing things, perhaps significant things.  Again, there is no way for you to know.  This is lost knowledge once again.

This is why TDD must be adopted as a team discipline.  Everyone must be on board and sufficiently trained to do it right.  If even one developer is not willing or able to adhere to the process, then TDD simply won't work.  It's got to be everyone, and it's got to be everything.

The key word to all of this is everything.  Anything less than everything isn't everything, and you lose the value of the comprehensive confidence that TDD offers in your process, your design, your institutional knowledge, and your product overall.

So, for my part, I choose to specify everything using tests no matter how trivial it may seem to be at the moment.  I think it's worth it.  Maybe you will too.

I hope you've found this series illuminating and helpful. Watch this space for more.


Posted on: November 30, 2022 11:55 AM | Permalink

Comments (6)

Please login or join to subscribe to this item
TDD is also useful outside of software development. I used TDD to prepare for my PMI-ACP exam.

I started taking simulation exams and studied where I had gone wrong until I could pass the exams.

Yes, good insight. In general I think a lot of what I've learned to make me a better developer has helped me in other parts of my life. The concept of a design pattern, analysis techniques like Commonality-Variability analysis, and TDD for sure,

Dear Scott
Very interesting the 3 articles you wrote about TDD
Thank you for sharing and for your opinions.

Dear Stephane
Your comment is very interesting
I'd like to see you manage projects using agile development approaches once you've learned how to pass the exam with that approach

You're welcome, Luis. Glad to provide some useful ideas.

Dear Luis, I managed a three-year project with over eight Scrum teams before I started on my ACP journey.

Please Login/Register to leave a comment.