Project Management


From the Sustainable Test-Driven Development Blog
Test-driven development is a very powerful technique for analyzing, designing, and testing quality software. However, if done incorrectly, TDD can incur massive maintenance costs as the test suite grows large. This is such a common problem that it has led some to conclude that TDD is not sustainable over the long haul. This does not have to be true. It's all about what you think TDD is, and how you do it. This blog is all about the issues that arise when TDD is done poorly—and how to avoid them.

About this Blog


Recent Posts

Acceptance Tests: Why Bother?

Do I Really Have to Test Everything? (part 3)

Do I Really Have to Test Everything? (part 2)

Do I Really Have to Test Everything?

TDD Tests as “Karen”s

A question that we are often asked is: “What is the difference between Acceptance Test Driven Development (ATDD) and Test Driven Development (TDD)?” These two activities are related by name but otherwise seem to have little to do with each other. 

ATDD is a whole-team practice where the team members discuss a requirement and come to an agreement about the acceptance criteria for that requirement. Through the process of accurately specifying the acceptance criteria -- the acceptance test -- the team fleshes out the requirement, discovering and corroborating the various assumptions made by the team members and identifying and answering the various questions that, unanswered, would prevent the team from implementing or testing the system correctly.

The word acceptance is used in a wide sense here:

  • The customer agrees that if the system, which the team is about to implement, fulfills the acceptance criteria then the work was done properly
  • The developers accept the responsibility for implementing the system
  • The testers accept the responsibility for testing the system

This is a human-oriented interaction that focuses on the customer, identifying their needs. These needs are specified using the external, public interfaces of the system. 

TDD, on the other hand is a developer-oriented activity designed to assist the developers in writing the code by strict analysis of the requirements and the establishment of functional  boundaries, work-flows, significant values, and initial states. TDD tests are written in the developer’s language and are not designed to be read by the customers. These tests can use the public interfaces of the system, but are also used to test internal design elements. 

We often see the developers take the tests written through the ATDD process and implement them with a unit testing framework.

Requirements from the customer

Before we continue, we need to ask ourselves -- what is a requirement? It is something that the customer needs the system to do. But who is the customer? 

In truth, every system has more than one customer... dozens at times:

  • Stakeholders
  • End users, of different types
  • Operators
  • Administrators (DB, network, user, storage)
  • Support (field, customer, technical)
  • Sales, marketing, legal, training
  • QA and developers (e.g., traces and logs, simulators for QA)
  • etc...

All requirements coming from all of these different customers must be addressed, identified and expressed through the ATDD process. For example:

  • The legal department needs an End User Legal Agreement (EULA) to be displayed when the software is first run, and for the end user to check off the agreement before the system can be used.  This is of no interest to the end users (who we sometimes think of as ‘the customers’), in fact might be an annoyance to them, but is required for the system to be acceptable to the lawyers.
  • The production support team needs all error messages in the system to be accompanied by error codes that can be reported along with the condition that caused the error.  Here again, end users are not interested in these codes, but they can be crucial for the system to be acceptably supported.

And let us not forget the the developers are customers too, who else do we build tracers and loggers for? This is an obvious, publicly visible facet of the developer’s work. But when do we need these facilities? When we try to fix bugs. When we want to understand how the system works. When we work on the system for any reason.  

In other words, when we do maintenance to the system.

Maintainability is a requirement

We need our maintenance to be as easy as possible. No car owner would like to disassemble the car’s engine just to change a windshield wiper; nor would they want to worry that by changing a tire they have damaged the car’s entertainment system. 

Maintainability is a crucial requirement for any software system. Software system maintenance should be fast, safe and predictable. You should be able to make a change fast, without breaking anything, and you need to be able to tell me reliably how long it will take. We expect this of our car mechanic as well as our software developer. So although maintainability is primarily the concern of the developer it definitely affects the non-technical customers. 

The way maintainability manifests itself in software is through design. Design principles are to developers as mathematics is to physicists. It’s the basis of everything that we do. If we do not pay attention to the system’s design as it is developed,it will quickly become unmanageable. 

How often, however, have you seen “maintainability” as a requirement? We’ve never seen it. We call it the “hidden requirement.” It’s always there but no one talks about it. And because we don't talk about it, we forget it about it; we focus on fulfilling the written requirements thinking that we will be done when we complete them. And very quickly, the system turns very hard and unsafe to change.  We are accumulating technical debt, which we could just call “the silent killer.” 

If maintainability is such a crucial requirement, where is the acceptance criteria for it? Who is the customer for this requirement? The development team.  

We need to prove to the customer that the design as was perceived was implemented, and that this design is in fact maintainable, that the correct abstractions exist, that object factories do what they are supposed to, that the functional units operate the way they should, etc... 

Indeed there is something that we can do, in the developers’ own language -- code -- that does precisely these things. It’s TDD. 

One key purpose of TDD is to prove and document the design of the system, hence proving and documenting its maintainability.

TDD is developer-facing ATDD

ATTD is about the acceptability of the system to its various customers.  When the specific customer is the development team then the tests are about the acceptability of the system’s design and resulting maintainability.  Our focus in this work is acceptability in this sense: is the design acceptable?  Is our domain understanding sufficient and correct?  Have we asked enough questions, and were they the right ones?  A system that fails to meet these acceptance criteria will quickly become too expensive to maintain and thus will fail to meet the needs of those who use it. 

Software that fails to meet a need is worthless.  It dies.  So, here again, failing to pass the “maintainability” acceptance criteria is the silent killer.  TDD is the answer to this ailment. 

Note to readers: This was a philosophical treatise. Specific, practical examples abound and will constitute much of our work here, so, read on.

Posted on: February 12, 2021 02:21 AM | Permalink

Comments (1)

Please login or join to subscribe to this item
I used to advocate TDD for beta version of project or the projects in production. First time, coming across ATDD and correlating the complaints from the scrum team of User stories maturity. Sometimes it takes for us 6 to 8 sprints on finalizing Acceptance criteria and Acceptance testing methods. may be gap in SM & PO expertise to process flow & work instructions documents. This article will help me to coach the team in better way on retrospect meetings.

Please Login/Register to leave a comment.


"If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas."

- George Bernard Shaw