Project Confusion in Transition Management

This article continues from my notes on acceptance testing in my region. The acceptance report or UAT Report as is typically known, is outlined here —

A UAT Report may be progressive such that it presents the summary of acceptance tests that are in progress at any time a report is needed, or a summary of concluded acceptance tests. It records the aggregation and or finality of test results, observations, discrepancies/deviations from the RTM, and resolutions if any that were made during the course of testing. 💯

The report may include out of scope testing such as tests or retests brought forward from a previous UAT regime. Or mention of (significant) outcomes from testing beyond formal test cases.

The final report is usually signed-off by the parties involved. The document may serve as a binding document that the planned tests are conducted, and present plans i.e agreement for completing whatever else that may be outstanding after the tests.

A UAT Report may comprise contents e.g.

A UAT Report in essence-

• Presents a consolidated view of what has transpired. A historical record on the build (where possible); especially when things were made on the fly and lacking as-built documentation.
• Is applicable to everyone- and produced by anyone- with vested interest in the testing or project. Anyone authorized can contribute to the construction of the report. 
• Emphasizes the terms to close a project phase (stage or work component) and move on to the next. The terms may include newly negotiated terms to deal with doubly unknowns uncovered during testing.
• Plan it as the end goal in testing. Exploit it as preparation, but to publicize structures and processes leading to the conclusion of testing. 

How do we create an acceptable UAT Report?

• Gather evidence of work outcomes as project progresses e.g. to ease documentation.
• Build upon a template that everyone is agreeable with. An approach visible to everyone is to evolve the report, editable by authorized users on a shared repository.
• Involve stakeholders that matters in crafting and reviewing the report, both formally and informally. As the saying goes, engage stakeholders early on in testing ('the project'), and bring onboard influencers in completing the testing.

References & past notes on the same topic:

• User Acceptance Tests 
• User Acceptance Tests (documentation issues)
• Notes on Documenting Acceptance Tests
• Doomed from the start: the importance of developing a sound test plan (2001)

.

This blog extends my previous notes on what may constitute a document for acceptance tests. The points are brief, and may not reflect what is practiced in your region or project.

Typical information required:

1. Indicate the rational for each test case or that for the test suite - specifically to ascertain compliance to specified or contracted requirements.
2. State how to perform the test for each test case
3. Present the expected outcome from the test case
4. Capture the actual result of each test case
5. Record observations and comment -on deviations from expected outcomes.
6. Attest the test results.

The acceptance test document may include appendices to conclude the test: 

• Evidence of completion of prerequisite tests 
• Evidence of readiness for testing e.g. prerequisite tests were conducted satisfactorily.
• Evidence of calibration e.g. the appliance and-or test tools were recently calibrated and readied for testing.
• Evidence of license - if any is required from the authorities or license owners.
• Statement from the customer - is prepared and ready to accept the deliverables (e.g. on completion of testing with positive results).

There was an incident where the external auditor of the customer, questioned the unfilled frames and scribbles outside of fill-in frames on hardcopy test forms. This occurred despite the testers having been briefed on protocols. The problem here is that incomplete filling and off-witted remarks on a formal test document can cast doubt or uncertainty on the test outcome. 

As such I would emphasize the inclusion of a chapter or section on procedures for completing test cases, i.e. specifically on executing specified tests and filling-up test case forms. Regardless of whether the forms are hardcopies or digital (such as using a purpose driven app on a tablet). I would place this as the first section of the test procedures. It should define information e.g. as follows –

• Capture pertinent details of personnel involved in the tests.
• Provide guidelines on acceptance testing policies, and procedures in completing test cases (note specified test cases are defined in a separate section). This section may include an overview of processes leading towards ratifying the test results (the reason being that each test set or group may have different requirements).
• Sign-off that the chapter has been read and understood before proceeding with the tests.
• Other information may include processes for negative test results (given that time is the essence for work at this stage).
• Additional information could be to indemnify the parties involved in the tests (note that some tests may be conducted live e.g. running hot, a module from DevOps).

See also my notes for training on –

• User acceptance testing
• User acceptance testing - documentation issues
• User acceptance testing - documentation 

The Handover and Takeover process (HOTO as is known affectionately in my circles) i.e. handling-over and taking-over of works affects all of us. These are notes on HOTO for my team newbies :-P

HOTO happens when:

• Someone substitutes a team member. This include when a project manager takes over a project that is already running, or steps in to run a recently secured project.
• A project deliverable is ready for delivery.
• The project ends and transitions to another state.

IMO the most problematic aspect of HOTO is taking over a project; for that matter even taking over an ongoing project activity. The reason being the outgoing incumbent is in the know, whereas the incoming manager is in the unknown. And is unlikely that everything will be made known. The brief below, focuses on the taking over process.

The following steps generalize what should occur when taking over a project, particularly:

1. Size-up the project that you will be taking over – work with the sponsor and project team. Identify outstanding work and qualify whatever may be unknown.  Knowing what is unknown is crucial, especially when you are taking over a troublesome project. Even if it seems to be business as usual, there will be matters that are not documented and may even be hidden. This is where risk and quality assessments will come in handy.
2. Determine the status of the project – the ① present status and ② upon taking over. This will be your assessment of the project status e.g. progress, performance, rates, etc. moving forward - and not that as was assessed in the past.
3. Demarcate responsibilities and accountabilities prior-to handling over and after taking over e.g. who and how would unknowns be handled. Clarify uncertainties. Note that fact finding (from seizing-up the project) is iterative and takes time.
4. Set expectations with all stakeholders, particularly those who may be most affected by the change. This will require time and effort to get to know the stakeholders, and to work with them on what to expect from henceforth. 
5. Inform stakeholders of your expectations.

Similarly, when handling over a project, work with whoever is coming onboard to agree on the status of project, complete all outstanding tasks, and update all project documents. That way you’ll mitigate whatever work you have done in the past from coming back to haunt you.

As with all guidelines, you will have to adapt recommendations of best practices, past experiences, and lessons learned to craft your own processes and procedures. The table below exemplifies common tasks which should be completed throughout the HOTO period:

Exemplify typical tasks before, during, after HOTO

When a project transitions to operations, the project team may have to supply operations or supporting documents (click here) to facilitate smooth operations.

There are two scenarios which can get us caught unawares when a project task or activity is delayed. It is either ① we failed to manage and monitor the activity, or ② there isn’t one or an adequate risk response plan for that activity. Taking over an activity that is delayed is to firefight - the implications are on the project team, the organization.

If we failed to track progress of the activity, then it could be construed as negligence; perhaps due to incompetence or ignorance e.g. that there is no succession planning (known unknown) or no certified professionals doing the job (unknown known). If we don’t have a risk response plan preferably in writing or in our heads or have some clues, then we would have totally failed to implement risk management.

Granted that events can happen unexpectedly – we should nevertheless have been on top of things. To begin with - there should be some form of risk management plan as the project kicks off.

Here’s my summary - when a risk event occurs, broadly:

Here are some principles to follow:

PMBOK® Guide – Sixth Edition (2017)

Past notes

• It Is About People
• Ignore Risk at Your Own Peril
• Risk on the Fly

This article continues from my notes on acceptance testing in my region. A broad view on documenting acceptance tests.

There are different ways to manage documents when testing for compliance to contracted terms (specifically with reference to the RTM). Given that at the end of the project, there could many test documents whether digital files and-or in hardcopies, generated for review, reference and filing.  One common approach is to log everything as a single file, and the other is to use multiple files.

Generally, there are two parts that need validation from stakeholders, particularly the customer – on the test procedures, and thereafter on the results from executing the test procedures. Hence a test document, would comprise minimally the test procedures and the means for recording the results. Taking it further, there may be a section to ratify the test results, and other sections for whatever else that may be included.

Logging everything into a single file may apply for the entire project (complex or otherwise)! Or may apply for a cluster of project components – let’s say. As hardcopies it would produce a single document with a single index that could span across multiple physical folders. And there would be hand writings and signatures on the hardcopies.

Logging multiple files is straightforward, i.e. one test document for each component or part. These may go into a single physical folder or span across multiple physical folders. Or each test in its own folder. The difference here is that information such as on test procedures is generally replicated, and each file will have its own file index.

Logging acceptance tests as digital files is similar as to logging hardcopies. For certain, there are better perks in logging digitally. Details on logging digitally in another blog.

Sounds straightforward ya. But what if for whatever the reasons may be, someone decided that the test procedures and test results should be filed separately. And (worse) there should be a file for each component or part. What could happen?

Here's a list of possible outcomes, from my recent and past experiences:

• Files inadvertently, even expectedly went missing because there were no proper, decisive, and thought-through means to relate, link and index the various test files together.
• Revisions were made to test procedures and related documents for subsequent tests without careful consideration for tests already conducted and filed in the past. Although the test intent may be the same, the approach and document formats are now different.
• The project is specified to conclude as a whole project. Understandably, any negative test result would impact the delivery of the project. By (re)specifying test reporting as multiple submissions (whoever did ..), the supplier/contractor argued successfully, that positive test results should be accepted as-is, and that the faulty component is independent of the rest. The point being the fault lies in a redundant portion of a system. Even though the system test should have resulted in an impact, the case was not – it was accepted that the system can perform and has performed accordingly to specifications; just that a component (or few) was down which can be and was easily ratified.
• Much time is required to search, corroborate, update and validate data or information, when similar purpose records are filed in multiple files. Imagine if the files are held by different individuals and not updated on time to a common repository.
• A deemed-independent test or case file may be closed in itself; but it may have impact on future activities in both technical and non-technical aspects. This can create issues in the long-term, which may surface after the project completes. Keeping pertinent case files alive such as in one collective file (for ease of reference/retrieval etc.), can reduce problems such as (non-apparent) issues being side-lined or neglected as the project march towards completion.

My take on documentation; including test documents —

1. plan your project documentation journey as early as possible in the project;
2. sleep over your document drafts, solicit inputs and revise them for their purpose. Whatever the approach may be, at the end of the day, IMO it will be the terms and how the terms are interpreted, which will dictate what project and acceptance test documents are like.
3. also read this