User Acceptance Tests (documentation issues)

Email Notifications

SHARE
Request to reuse this

This article continues from my notes on acceptance testing in my region. A broad view on documenting acceptance tests.

There are different ways to manage documents when testing for compliance to contracted terms (specifically with reference to the RTM). Given that at the end of the project, there could many test documents whether digital files and-or in hardcopies, generated for review, reference and filing.  One common approach is to log everything as a single file, and the other is to use multiple files.

Generally, there are two parts that need validation from stakeholders, particularly the customer – on the test procedures, and thereafter on the results from executing the test procedures. Hence a test document, would comprise minimally the test procedures and the means for recording the results. Taking it further, there may be a section to ratify the test results, and other sections for whatever else that may be included.

Logging everything into a single file may apply for the entire project (complex or otherwise)! Or may apply for a cluster of project components – let’s say. As hardcopies it would produce a single document with a single index that could span across multiple physical folders. And there would be hand writings and signatures on the hardcopies.

Logging multiple files is straightforward, i.e. one test document for each component or part. These may go into a single physical folder or span across multiple physical folders. Or each test in its own folder. The difference here is that information such as on test procedures is generally replicated, and each file will have its own file index.

Logging acceptance tests as digital files is similar as to logging hardcopies. For certain, there are better perks in logging digitally. Details on logging digitally in another blog.

Sounds straightforward ya. But what if for whatever the reasons may be, someone decided that the test procedures and test results should be filed separately. And (worse) there should be a file for each component or part. What could happen?

Here's a list of possible outcomes, from my recent and past experiences:

• Files inadvertently, even expectedly went missing because there were no proper, decisive, and thought-through means to relate, link and index the various test files together.
• Revisions were made to test procedures and related documents for subsequent tests without careful consideration for tests already conducted and filed in the past. Although the test intent may be the same, the approach and document formats are now different.
• The project is specified to conclude as a whole project. Understandably, any negative test result would impact the delivery of the project. By (re)specifying test reporting as multiple submissions (whoever did ..), the supplier/contractor argued successfully, that positive test results should be accepted as-is, and that the faulty component is independent of the rest. The point being the fault lies in a redundant portion of a system. Even though the system test should have resulted in an impact, the case was not – it was accepted that the system can perform and has performed accordingly to specifications; just that a component (or few) was down which can be and was easily ratified.
• Much time is required to search, corroborate, update and validate data or information, when similar purpose records are filed in multiple files. Imagine if the files are held by different individuals and not updated on time to a common repository.
• A deemed-independent test or case file may be closed in itself; but it may have impact on future activities in both technical and non-technical aspects. This can create issues in the long-term, which may surface after the project completes. Keeping pertinent case files alive such as in one collective file (for ease of reference/retrieval etc.), can reduce problems such as (non-apparent) issues being side-lined or neglected as the project march towards completion.

My take on documentation; including test documents —

1. plan your project documentation journey as early as possible in the project;
2. sleep over your document drafts, solicit inputs and revise them for their purpose. Whatever the approach may be, at the end of the day, IMO it will be the terms and how the terms are interpreted, which will dictate what project and acceptance test documents are like.
3. also read this