PMI Global Insights

By: Scott L. Bain

A developer was working for a large financial company. The company leaders had an idea for a new product that would allow their customers to participate in foreign stock exchanges without having to stay up all night to time their transactions.

There would be many things to develop. A website that allowed customers to enter buy, sell, limit, and margin orders throughout the day to be transacted later when the foreign markets were open. A database would store all these accumulated transactions. Marketing would have to come up with a plan to promote this new product. Also a component, running on the server, would trigger at midnight to access a web service and commit these transactions as a batch.

The developer in question was assigned to create that timed component. The requirements seemed rather simple: The component would run on the main server and watch the system clock.  At midnight it would activate, retrieve all the transactions that had been stored throughout the day, open a connection to the external service, commit the transactions in the order they were entered, receive a confirmation code from the service, and write a log entry into the company’s daily activity log that included that code.

They knew how to open that web service connection, what the correct logic was to “commit” a transaction, and how to access the company’s logging API. Of course, they knew was “midnight” was. They met with the database administrator to learn the exact nature of the SQL table that would hold the transactions and what key to use to retrieve them in order. All of this seemed relatively familiar and straightforward and so they gave a low estimate on time and effort needed to get the work done.

Indeed, they got the component finished well within that time frame and checked the code in. The website and marketing were still in development and so they moved on to other work.

A few weeks later, the rollout of this new service was scheduled for a Monday. The day arrived and the product launched. It was immediately popular; many transactions were entered all throughout the day and everyone was enthusiastic about the product. The developer went home at the end of the day confident that the component would trigger at midnight and process the orders.

In the wee hours of the morning, they received a text. Disaster occurred; everyone was being called into the office. None of the transactions had gone through and they were anticipating many unhappy customers, perhaps even some legal action taken against the company. The developer went into work feeling a sense of puzzled dread.

What had gone wrong?

The transaction batch did not commit on time. It was too late. An SEC rule stated that if not every transaction was committed then none of them could be as a matter of fairness, so the external service had rejected the entire batch. Massive financial implications were the result.

This developer was relating this story to me a couple of years later.  I knew this person because they had been a student in several of my courses and in them had studied design, analysis, and testing.

“I didn’t know there was a deadline, a window past which the transaction would fail,” they said.

“An unasked question; that’s often the problem,” I replied. 

“Sure,” they said, “but then I realized something.  Even if I had known what that window was, I had accepted the requirement that the batch commit had to happen at midnight.  At midnight.  Nothing can happen at midnight.  Midnight is an infinitely short period of time and computers are not infinitely fast, and neither are internet connections.  Once it is midnight, it is immediately after midnight, right?  So what did the requirement actually mean?  By midnight?  Then how long before midnight should or can I start?  Starting at midnight?  Then how long do I have?  Also, this SEC regulation about all-or-nothing was something I had no idea about.”

“It sounds like this was not your fault,” I said.

“I guess not,” they replied, “but I realize that now, today, the way I work would have prevented this.  Today I would be driving everything from tests and the tests themselves would force me to ask the questions I didn’t.  The acceptance tests would make me ask ‘how will I know if I’ve failed?’ because I would need to start with a failing test.  Also, I would have realized the SEC was a stakeholder to my work and I would have included someone who understood their perspective.  The unit tests would force me to think carefully about the design of the component itself so I could do things like test it without having to execute the test at midnight in my pajamas.”

“Separation of concerns,” I said.

They smiled.  “Exactly. I now know how to achieve that without overcomplicating my designs.  But the real crux here was the fact that I didn’t understand what the requirements really meant, because the communication and collaboration had failed.  I lost that job as a result.”

This is not uncommon, unfortunately.  Communication, especially between technical and non-technical people is extremely unreliable.  Even when everyone intends well, there are too many mismatches among perspectives and too many questions that are missed.

Acceptance Test-Driven Development (ATDD) seeks to address this problem in a fundamental and comprehensive way.  How can we achieve a complete, meaningful, shared view of the product that is being proposed?  How can we record the results of the collaboration in a way that can be validated against the actual work as it is completed?  How will we know when we’ve succeeded?  How will we know when we’ve failed?

ATDD is a framework for highly effective collaboration across all parts of an organization.  It produces tests, but is not a testing activity.  It is a way to ensure that a product’s requirements are fully understood, including everyone’s perspectives. 

It is not a difficult framework, but it is essential if you want to ensure that everyone is on the same page and that the work to be done will be in complete alignment with the business value that drives it. It is something that everyone needs to know how to do.

Join me at PMI Training 17-18 April for NEW! Acceptance Test-Driven Development: Improving Communication, Empowering Collaboration, and Creating a Shared Specification for New Products with Scott Bain (pmi.org). 

Interested in learning more about trainings specifically developed by project management experts for project managers in a small group setting? Check out the PMI Training website for upcoming offerings.