Sustainable Test-Driven Development

Acceptance Test-Driven Development (ATDD) is a collaborative activity where individuals from different parts of an organization, both technical and non-technical people, get together to create acceptance tests before development begins.  It is not without cost, of course, and so many might ask if it is worth it and why.

A story will help to illustrate why ATDD is so important.  This is a real-world example.

It involves a relatively small software development company in Montana. Its main client was a statewide provider of satellite dish hardware and video services in rural areas of the state. In these areas it is too expensive to lay out cable or fiber optics and given that there were so few customers the big companies like Comcast and Frontier were not interested in the business.  So a local company served this customer base.

This boutique software shop had written custom applications for all of the company's activities including general ledger, payroll, installation and maintenance scheduling, and so forth. One of these applications had to do with the documents that their customers had signed when they contracted for satellite dish services. These contracts were digitized and stored in a database so they would be available if there were any disputes or conflicts about level of service agreements, payment schedules, or anything that the customers had agreed to when they signed up.

A new requirement was issued. A feature was to be created that could be activated daily to scan all of the documents stored in the database and delete those that were more than 7 years old. The developers accepted this requirement as stated and considered it to be a fairly trivial task. Accordingly they gave a very low bid and promised completion in a short period of time.

They created a plugin component that could be installed in the database server's workflow that would activate at a convenient time during the day, when business was slow, scan all the documents in the database, comparing their date field with the system clock, and erasing those that were more than 7 years old.  As promised they got this done very quickly.

What they did not do was ask why. Why did these documents need to be deleted?

Why didn't they ask? Because they thought they knew the answer already, that it was obvious. They assumed that the reason the old contracts should be deleted was that they were out of date anyway, had little to no value, and were filling up the database which would consume resources and impede the performance of the system. 

This requirement was one of many that had been issued and when everything was complete a meeting was called so that they could deliver the products, review them, arrange for installation, and prepare for the next iteration of work. At that meeting were several individuals that had not been present when the requirements were accepted.

When they got to the component in question they explained that it would serve to free up space in the database and improve the performance and responsiveness of the applications. One person at the meeting, a legal expert, spoke up and said “that's not the reason we wanted the documents to be removed.” The reason the documents had to be erased was a new regulation from the state of Montana that said that you may not keep a person's signature for longer than 7 years, neither in physical nor digital form.  It was about privacy and identity protection, and reflected the view of that conservative legislature about individual rights.  This had caused banks all across the state to discard their signature cards when they got sufficiently old, and have their customers come in to create new ones.  But this law did not apply only to banks.

The development lead said “well maybe we did it for the wrong reason but we did the right thing. Those documents will be erased and therefore you will be in compliance with this regulation.”

The database administrator was also at the meeting. They sighed and shook their head and said “no, that won't work at all.  We’ll still be in violation.”

If you are responsible for a database one thing you will do if you want to keep your job is back it up. So the database administrator did incremental backups every day of all of the information in the database including the digitized contracts. Those incremental backups were stored off site at another location which was also in Montana. Those backups would contain the documents and the signatures spread throughout the incremental history of the database. That would violate the law.

Removing data from incremental backups is extremely complex. It involves creating a kind of recursive worm that can make its way incrementally through all the different versions of the backups, seeking out the old documents and erasing them. It would take much more time and be far more expensive than the simple component that the developers had created. They gave an estimate for this and the company said that it was far too costly. Also it would be very complex code, might not work properly at first, would probably require several iterations to get right and the deadline was looming.

They decided instead to re-architect the entire system. They would have two databases: one for their general ledger and other business records that was backed up normally, and another, separate database for all of their scanned documents. That database was not backed up because if you consider the “why” of backing something up, which of course is to ensure that information is never lost, there's another way to accomplish that. They decided to use data replication in that case. Another server at another location would store the same document data and would synchronize with the main server throughout the day. When you delete something from a server that is replicated the deletion gets replicated automatically so there is nothing left to do

This was still a fair amount of work of course and it was work that no one anticipated in the estimate because no one had spoken to the legal expert who understood the “why” of the contract deletion, and the DBA had made the same assumption that the developers had.

Communication is difficult, to say the least.  But it is also crucial when we create anything collaboratively, which almost all software is today.  Accepting a requirement involves critical thinking from many different points of view or important details will be missed or misinterpreted, resulting in wasteful work and costly errors.

ATDD is a framework for structured conversations that can reveal the real nature of requirements in a way that can be understood and confirmed by all involved.  It also records that nature in a way that can be verified against the system at any time, because it produces executable tests.  It is not difficult to do once everyone is properly trained, and it addresses a major cause of failure in software development: rampant misunderstandings.

See this page for information on our ATDD course:

https://www.pmi.org/business-solutions/agile-training/technical-solutions/acceptance-test-driven-development