Project Confusion in Transition Management

This article is a compilation of notes on acceptance testing in my region.

Acceptance tests are conducted when a system or deliverable is defined as ready for acceptance. How a deliverable is determined as ready or complete is a matter for agreement between the project owner i.e. customer and the project team aka supplier. Generally the completeness of a deliverable should cover all aspects of contracted requirements. People's expectations in particular and specified granularity on requirement details for quality matters, can influence and change what will ultimately constitute the completeness of a project deliverable.

Acceptance tests are conducted in the presence of the customer, typically after the customer has agreed to take over a deliverable.

Testing considers the location and are conducted at both the following (always for large projects):

• Staging site which may be at the Factory where a system or product is created, or at a location where the system or product is setup prior to delivery to the customer's site.
• Actual site where the project is deployed

In brief, the following are performed leading to the actual test (see test categories):

1. Agreeing on what will constitute acceptance testing for that test category - especially if not already specified or detailed.
2. Document test - review test procedures and walkthrough the steps, a dry run to test a deliverable, system or product.
3. Preliminary test - a real test conducted with or without the presence of the customer to validate the tests and readiness for acceptance testing. This will be the last save-face stage or opportunity where flaws are affirmed and finally rectified.
4. Conduct actual or final test e.g. Site Acceptance Tests

The different test categories in sequence:

1. Factory Acceptance Tests - this exclude testing at subcontractors', suppliers' as they are not known to your customer
2. Site Acceptance Tests - that whatever delivered from Factory are as-is. Functional tests may be conducted separately. 
3. Site Security Acceptance Tests or Systems Security Acceptance Tests - may involve separate tests such as physical security, checks on personnel, and cyber security.
4. Reliability Acceptance Tests - this exclude the warranty period
5. Audits by independent internal and-or external parties - this exclude any visit by the authorities

The number of tests can easily double when we factor in requirement for preliminary testing (the exception being the reliability test itself and audits - we can't tell auditors what to do). And yes, there will be corrections in every test category.

Upon completion of each test from the category list above - at the least, will incur the following activities:

1. Review test results
2. Ratify test results
3. Massage the customer to Accept test results

Note that acceptance tests may be conducted:

• Before handling over and taking over (HOTO) the deliverable
• Throughout the HOTO process 
• After HOTO of a deliverable!
• Or in any manner as agreed between the customer and supplier.

A best practice approach to acceptance testing:

• Let the domain experts determine and decide on what to test for acceptance of deliverables.
• Corroborate testing with what was contracted.
• Seek the agreement and approval of all stakeholders especially the customer on the tests. The exception being - perhaps if you are the empowered authority or auditor who can barge in to test for compliance.

Read also User Acceptance Tests (documentation issues)

* I'll keep this as a live document. Hmm ..

I blogged about a missed opportunity previously. That missed opportunity was something which was not planned. Although the matter has been raised on a few occasions, including by myself throughout the course of the project. I sensed the core team was apprehensive in pursuing the positive risk – a story for another blog.

I’d like to detail a little of their wins. The project team was able to accomplish their mission within the prescribed scope and timeframe, broadly for the following reasons:

1. Consortium partners have their reputation on the line to deliver the project successfully. They are among the niche product suppliers and market leaders in their field. They also supplied the aging components of the old system.
2. The customer was determined to finish the project with priority, to expand the scope of coverage with enhanced technologies and in replacing a decades old system that is deteriorating and hard-to but not incapable of integrating with new regulatory systems fast coming online.
3. The people factor.

Project team members which include the customer, other stakeholders and authorities, were able to bond quickly to collaborate effectively, complement each other, and perform efficiently on the project. Despite the ludicrously lean staffing of the overall project team and that of the partners, for that immense size-value project. I observed the project team quickly settled into framing and abiding by the simple guiding principles below, no matter what transpires:

1. Stick to the agreed scope.
2. Deliver less when hard pressed to meet deadlines. In our lingo, 'work on the quality' but 'comply wholly to mandated scopes'. This occurred to internal customers of the project owner. In this aspect, there has to be willing recipients. I’d say, in cahoots lah
3. Adapt suitable Agile practices wherever possible in the contracted waterfall project.

The people factor matters most. Some contributing factors include:

1. Majority of the team members were seasoned executives. And almost all of them were promoted during the project, in their respective companies. (Incidentally all were males. For another discourse.)
2. Embracing the fact that work must be acceptable – the how-to, by when, in what form, and with whom.
3. Nobody works alone. The lean team works with contractors and contracting throughout. So, there is really no point in dissenting.

Hence in managing risk, manage people first. Know who they are, what they want, and how to define specific project goals. Together. Then commit to the work, to deliver the project.

Accept that project work is done only once.

And move on.

Most companies would jump at the opportunity to promote themselves, especially when the opportunity involves publicly accessible media. The project was completed successfully within contractual timeframe and delivered wholly as specified in the terms, with renewable maintenance of over 20 years. Everyone was gleeful. What a cause for celebration!

The project involves putting in place a high-availability, high-dependency, mission-critical infrastructure system. The highlight which was the integration of different sensor and scanning technologies to produce results, which previously were produced by a proprietary and purpose-built system (it is still the typical approach today mostly for that kind of need). The business succeeded in bringing together rival European and Japanese conglomerates to secure the contract. What more to shout about!

At the end of the project closing meeting, a by-the-way remark ‘we could do a press release of our achievements’ on the integrated technology and business collaboration amongst rivals. We even have a new acronym for a universally regulated system! That came from the customer. And they have an office for publicity. As you recall what my opening statement implied – the thoughts for publicity were quickly extinguished at the table.

So, what happened?

Without the fluff –

1. Managers were not keen to pursue additional work that were not specified earlier, and beyond the project timeline. The unplanned publicity work must be justified, completed quickly, and require repeated proof-reading, vetting, and multiple layers of approval.
2. Local experts were indecisive as to what were their technical accomplishments. They also lack the experience in publishing their work, to put it mildly. Even though a foreign partner has suggested on one occasion insistently, there is much to write for peer-reviewed journals.
3. There lacks a culture to write. IMO, to openly share.

The publicity could have been enormous through peer-reviewed technical papers and of course in the general media – and perhaps even claims of copyrights; but too late for patents if any is possible. Now, the lustre is no more.

What we can do –

1. Agree on the publicity beforehand. Commit to writing if needed. Plan for publicity – the contents as to what can and cannot be revealed. Besides – we need to plan for celebrations, and publicity is simply a quintessential component of project achievements.
2. Research on how to go about the publicity when in doubt. Present the facts – rational for publicity and sharing to all sponsors.
3. Appoint professionals to mitigate risks in matters of media or press releases. Yes, this adds to costs, but it is a necessity to say the least. The project should have had its occasion.

Have you encountered a similar incident?

(I may not respond in time but please indulge.)

The key to successful execution of project activities, lies with people & the information they have access to. This include the manner & channels of accessing information that would enable them to perform their tasks independently & confidently. The quality of information (depending on the context of work) is another contributing factor. Granted that there are restrictions & limits as to what & how information can be accessed & used.

This has to be planned.

In a nutshell, the Communications Management Plan outlines for the start of a project, a mutually agreed framework on the exchange of information. The means of which rely on the clarity of project stakeholders & their appointed roles. The communications plan defines for all project stakeholders:

• Who is assigned/authorized, to do what, in which role (on the responsibility assignment matrix; which may include a reporting hierarchy).
• What is to communicate, using what medium, by when.
• Where to feedback (e.g. on protocols governing a project activity), or escalate (typically on matters of performance/dispute) should communications fail.

Responsibility Assignment Matrix

The crux of the Communications Management Plan is the Responsibility Assignment Matrix. IMO. That outlines the framework for interaction, sharing of information among project stakeholders. Fig. 1 illustrates the responsibility assignment matrix at a high-level. You can use other models similar to RACI/RASCI, and tailor the matrix to your corporate needs. Drill down to sufficient granularity that befits your project.

Fig. 1 — RASCI Matrix

I would advocate crafting a simple Communications Management Plan before starting any work, and develop the plan progressively; especially if work involves multiple parties (particularly in a cross-cultural setting), is complex & takes time. A simple plan will suffice, nevertheless a plan that provides clarity on expectations & distributed to everyone on board.

A quick start to create a communications plan:

• Determine contractual communiques e.g. reporting, frequencies, logs on activities, & records of work performance.
• Identify required/necessary project documents.
• Clarify project roles & appoint information owners. 

Enforce the Communications Plan 💪

Incorporate vital communications plan protocols (at the least) into the QMS/QA to monitor compliance to specified/agreed baselines; which may include checklists for communications, logs, & artefacts from outcomes. Archives are useful for QA audits; that acceptable standards for communications are followed. Identify risks associated with communications to better manage & mitigate surprises. And conduct periodic reviews & updates to the plan, subject to established guidelines, to keep communications flowing. That would be to everyone's advantage.

P.S. Post-Pandemic

📢 The initial feedback on the viability of working from home & feasibility of hot-desking, has been encouraging. Most surveys including from my team on their experience during the lockdown, indicated that loss of performance in carrying out project activities has been insignificant. Loss of performance if any, is due mainly to taking cautious approach at work & in interacting with people in the field; & hiccups in using technology (which can be resolved easily through training & practice). I attribute the success to planning for communications early & on paper before closing the office. As work environments change post-pandemic, a well-thought-through Communications Management Plan for the company/project team can serve as the instrument to guide & govern associates working remotely or from home. 

Related snippets

• My previous topic
• https://tinyurl.com/ydye8bnq

This brief is going to read like course notes! I wrote about a lesson learned in ignoring proper risk appraisals & risk documentation in my previous blog.

I hardly hear nowadays — the words quality & risk used literally particularly when a project, stage or project phase duration is deemed short; unless quality & risk are explicitly specified in requirements. Work seems to be always in a rush, with little time to spare for thinking through what was planned the first time around. Working on projects feels a lot like routine operations. Couple with the fast pace in Agile, the notion that if it works one should just go with it again seems to be the new norm. Working like clockwork may help the team deliver any assigned project pronto, eliminate overall project risks, and minimize disruptions. But it presents a downside, among which include:

• Reliant on routines & safe zones
• Complacency
• Falling behind in exploiting new & emerging skills & technologies 
• Loss of innovation (capability)
• Eroding (team) professionalism

Whatever the circumstances may be to rush a project (or lagging project activities), give a thought for project risks. It is an essential management attribute of a quality project. A quick recap on dealing with project risks ~

Some structures. There has to be some form of structure when discussing risks. Consider a structured approach to administer project risks. For starters:

• Adapt guidelines & recommendations of a risk standard to the needs of your business or project. The standard may be from PMI or any other recognized bodies.
• Categorize identified risks e.g. high, low, medium; where possible put a value to the risk e.g. risk score or perhaps cost should the risk occur.
• Plan to mitigate the threat or exploit an opportunity; and put in place contingencies should the risk event happen. Keep in mind that contingencies for positive risks are meant perhaps to seize the moment e.g. secure pre-authorized approval to act. 

Give it some numbers. Create a risk score ႟. There are different means to evaluate risk. Choose a suitable approach /template e.g./wordings. Rate your risk attributes e.g.:

1. What is the likelihood the risk will happen
2. What will be its impact
3. How hard is it to detect the risk (easy ↓ hard ↑ )

႟ = Risk Likelihood ✘ Risk Impact ✘ Risk Detectable

I would advocate framing risk & quality agendas when discussing project activities; especially when hard pressed for time to examine & investigate associated risks & quality concerns in separate meetings. For each prioritized risk:

Remember ...

1. Assign risk ownership - as internal control & to flesh out the following points later
2. Plan risk responses i.e. risk mitigation; and
3. Implement risk responses (may entail changes to plans /activities at subsequent meetings), and monitor the risk response plan/activities (a QA task because the processes to monitor risk responses being put in place & in-effect are essentially quality processes)
4. Plan for contingencies
5. Test (where applicable) & review planned contingencies for identified risks. Depending on the context, risk contingencies may utilize evolving technologies which may silently run out of efficacy with knock on effects e.g. no parts, no expertise, no vendors. That will be really disastrous when you then have a useless contingency plan in hand. This brings us back to why we must revisit to review, revise & renew RA templates /samples /past submissions.

References

• Lukas, J. A. & Clare, R. (2011). Top 10 mistakes made in managing project risks. Paper presented at PMI® Global Congress 2011—North America, Dallas, TX. Newtown Square, PA: Project Management Institute.
• Hillson, D. (2014). Managing overall project risk. Paper presented at PMI® Global Congress 2014—EMEA, Dubai, United Arab Emirates. Newtown Square, PA: Project Management Institute.

"Documentation is not understanding, process is not discipline, formality is not skill." — Jim Highsmith, Adaptive Software Development