Risk Insights from The Risk Doctor

No-one knows the future with perfect certainty, which of course is why we need risk management. But sometimes we try to guess what might happen, and use that information as a basis for planning or decision-making. The proper name for such a guess is an “assumption”, and these are an important source of risk, for projects, businesses and life in general.

An assumption is a way of dealing with an uncertain future when there are a number of possible options. In its simplest form an assumption is a decision to proceed on the basis that one option will turn out to be correct and the others will not happen. For example, we might assume that our suppliers will deliver on time, or that our client will sign-off all approvals within two weeks, or that all key members of our project team will remain for the duration of the project. But what happens if we assumed the wrong thing? In most cases a false assumption would lead to a problem for the project, since we usually tend to assume that things will go the way we want.

Of course not all assumptions matter equally. There are some assumptions which might prove false without having a significant effect on the overall project, but there are others where a different outcome could be serious. Fortunately there is a simple process for testing how risky assumptions might be, and for including them in the risk process if necessary. A simple IF-THEN statement can be written for each assumption, in the form : 
“IF this assumption proved to be false, THEN the effect on the project would be …”

The IF side tests how likely the assumption is to be unsafe, and the THEN side tests whether it matters. Another way of describing this is to see the IF statement as reflecting probability, whereas the THEN phrase is about impact. And probability and impact are the two dimensions of risk. This simple approach can be used to turn project assumptions into risks. Where an assumption is assessed as likely to be false and/or it could have a significant effect on one or more project objectives, that assumption should be considered as a candidate risk.

This type of Assumptions Analysis is a powerful way of exposing project-specific risks, since it addresses the particular assumptions made about a given project.

There are however two dangers with this technique :
1.    The first weakness is that this technique can only consider explicit assumptions, which have been consciously made and openly communicated. There are however many implicit or hidden assumptions which we all make every day, some of which are very risky. 
2.    Secondly this approach tends only to identify downside risks, threats that a particular assumption may prove false and result in a problem for the project. Assumptions Analysis is not good at identifying opportunities because most of our assumptions are optimistic.

The first shortcoming can be overcome by a facilitated approach to identifying and recording assumptions, using someone independent and external to the project to challenge established thinking. To be fully effective, Assumptions Analysis needs full disclosure.

For opportunity identification, the technique can be extended to address and challenge constraints. These are restrictions on what the project can or cannot do, how it must or must not proceed. But some of these constraints may not be as fixed as they first appear – indeed some of them might be assumed constraints. In fact it might be possible for a constraint to be relaxed or perhaps even removed completely. In the same way that assumptions can be tested to expose threats, a similar IF-THEN test can be applied to constraints to identify possible opportunities: 
“IF this constraint could be relaxed or removed, THEN the effect on the project would be …”

Instead of making assumptions about the future, or accepting that stated constraints are unchangeable, being prepared to challenge assumptions and constraints can expose significant threats and opportunities which can then be addressed through the risk process.

The term GIGO is famous as an abbreviation for the phrase “Garbage In Garbage Out”. Originally used in the IT industry, it described the fact that the output from a computer system was only as good as its input. Even the best program cannot take meaningless data and produce meaningful results. Of course GIGO applies much more widely than just computers. The integrity of the output from almost every system or process depends on the integrity of its input – with the possible exception of the human brain, which seems able to create order out of chaos by the application of reasoning and intelligence (at least sometimes!). And “Garbage In Garbage Out” can certainly apply to the risk management process..

A recent variant on GIGO translates it into “Garbage In Gospel Out”. This describes the tendency of people to accept output from a system without judging it critically. Even if the input is rubbish, we still believe the result, usually because we don’t fully understand the way the system works to produce it. This is sometimes called “blind faith”. “Garbage In” to the risk process can mean lack of agreed objectives, poor or lazy risk identification, or use of inappropriate risk responses. “Gospel Out” means treating outputs as infallibly true, with no need for interpretation or judgement. 

There is of course a third meaning for GIGO – “Gospel In Garbage Out” – where the system takes good data but introduces errors or makes wrong calculations, and so produces nonsense results. In the risk process this often arises from lack of time, attention or resources for risk management, the use of inappropriate tools or techniques, or lack of risk skills.

How can risk management avoid these three GIGO problems? The third is perhaps easiest to address, since “Gospel In Garbage Out” can be avoided by using a sound risk process, together with staff training and proven tools.

Both “Garbage In Garbage Out” and “Garbage In Gospel Out” can be tackled by applying two filters to the risk process :

1. Verify the input. This means asking questions about the data fed into the risk process. Is it complete? Is it up to date? Can we trust it? Is it influenced by bias, assumptions or a limited perspective? Is it accurate? Is it relevant? And most importantly – is it true?
2. Validate the output. Here we are checking the results of the risk process to see if they make sense. Do the outputs match expectations (and if not, why not)? Are they counter-intuitive (and if so, why)? Is there a clear trend from previous results? Can we double check using other approaches? And can we act on the results with confidence?

Of course verification is not a simple task because input to the risk process is inevitably uncertain. It involves subjective judgements about what the risk is, how likely or severe it might be, and what responses are appropriate. But we should still ensure that input data quality is as high as possible.

And although risk outputs may sometimes be surprising or counter-intuitive, they should always make sense if the underlying risk process is sound. We should not be afraid to challenge assumptions and test outputs before we use them as a basis for decisions and actions.

So verifying input (“Is it true?”) and validating output (“Does it make sense?”) can protect against the perils of GIGO. These dangers are real but they can be overcome, and they should not stop us from using risk management on our projects or in our business. After all, there is one thing worse than GIGO, and that is NINO : “Nothing In Nothing Out”!

It is easy to understand why some people think that the risk response development phase is the most important part of the risk process. This is where we get the chance to make a difference to the risk exposure of our project. If we design and implement good risk responses to address the risks we have identified and assessed, we will be able to minimise threats and maximise opportunities, and so optimise the likelihood of achieving our objectives. But if our risk responses are ineffective (or not implemented), the level of risk exposure remains unchanged – or may even get worse!

But how can we tell if our risk responses are good enough? Can we assess their potential effectiveness before we decide to implement them? Here are seven “Grade A” criteria by which you can test whether your planned risk responses are likely to work. To be effective, all proposed risk responses should be:

1.    Appropriate – The correct level of response must be determined, based on the significance of the risk. This ranges from a crisis response where the project cannot proceed without the risk being addressed, through to a “do nothing” response for minor risks. We should not spend large amounts of time or effort developing aggressive responses for minor risks, but we must also not spend too little time considering how to deal with key risks.

2.    Affordable – The cost-effectiveness of risk responses must be determined, so that the amount of time, effort and money spent on addressing the risk does not exceed the available budget or the degree of risk exposure. Each risk response should also have an agreed budget, added to the approved project cost plan.

3.    Actionable – An action window should be determined, defining the time within which risk responses need to be completed in order to address the risk. Some risks require immediate action, while others can safely be left until later. We must be careful not to leave it too late before we act.

4.    Achievable – There is no point in describing risk responses which are not realistically achievable or feasible, either technically or within the scope of our capability and responsibility. If your planned response is “Hope for a miracle” or “Invent a radical new solution”, you may be disappointed! 

5.    Assessed – All proposed risk responses must work! The “risk-effectiveness” of a response is best determined by making a “post-response risk assessment”. This assesses the level of residual risk assuming effective implementation of the response, including secondary risks of course. The situation after implementing the risk response must be better than before!

6.    Agreed – The consensus and commitment of relevant stakeholders should be obtained before agreeing responses, especially if the proposed response might affect a part of the project in which they have an interest.

7.    Allocated & Accepted – Each risk response should be owned by a single person (and accepted by them) to ensure a single point of responsibility and accountability for implementing the response. Allocating risk responses requires careful delegation, including provision of the necessary resources and support to allow effective action to be taken.

Each proposed risk response should be assessed against these seven criteria before it is accepted. A “Grade A” response will pass all these tests, and is more likely to achieve the desired effect than a response which has not been properly considered or evaluated.

Risks can be identified and described at different levels of detail, and there can be considerable variation between different projects or organisations. Some projects identify just a small number of high-level risks, while others have many hundreds or even thousands of detailed risks. A generalised or high-level description of risk can make it difficult to develop responses and assign ownership, while describing risks in a lot of detail can create a great deal of work. How can we determine the correct level of detail? There are three components to consider : management, ownership, and reporting.
1.    Firstly, risks should be described at the level to which they are will be managed. A high-level description such as “Something unexpected might happen during the project” is quite useless as no management action is possible at this level. Too much detail is also pointless, for example “George Smith the junior system architect may break his right leg at the football match next Tuesday night and not be able to finish the Phase 2.4.2 detailed design drawings.” The risk might be better stated as “Key staff may not be available when required to complete the system design.” At this level the risk can be managed proactively, with careful resource planning, use of shadowing or deputies, and ensuring that key tasks are not assigned to one person. Of course it is true that some risks will need to be managed at a detailed level while others can be addressed at a higher level.
2.    Secondly, each risk should be described at a level of detail where it can be assigned to a single owner, with clear responsibility and accountability for addressing the risk. However this also allows for some variation in the level of risk description, as risk owners can range from junior team members who might be responsible for detailed risks, through to the project sponsor or senior managers who are only interested in the higher level.
3.    Thirdly, the level of risk description should match the reporting needs of the person receiving the risk report. Project team members need detailed risk descriptions for those risks which they are responsible for managing. The project sponsor or client needs less detail, perhaps with groups of risks being summarised into high-level descriptions.

Each of these three answers suggests that risk descriptions can be useful at various levels for different purposes. There is no one right level that meets all needs. So what can be done?

One useful tool addressing this issue is the Risk Breakdown Structure (RBS), which is a hierarchical structure describing sources of risk to the project. This allows risks to be described at increasing levels of detail throughout the project. At the top level (Level 0), all risks are simply “Project Risks”. But this can be broken down into major sources of risk at Level 1, such as Technical Risk, Commercial Risk, Management Risk, External Risk. Each of these major areas can be further detailed at Level 2 (for example Technical Risk could be subdivided into Technology, Performance, Reliability, Interfaces etc). At the lowest level individual risks are described under each specific source.

Different RBS levels can then be used for different purposes. Detailed risk reporting, ownership and management can take place at the lowest level. Higher RBS levels allow groups of risks to be rolled-up and summarised for reporting, ownership and management further up the organisation. So the project safety engineer may need to know about a specific risk affecting a particular product trial (RBS Level 4), whereas the company's Chief Technical Officer may be interested in the overall level of technical risk on the project (RBS Level 1).

Risk descriptions at different levels of detail are useful in different ways. Instead of insisting that all risks are described at a single level which may not suit all needs, using a hierarchical RBS can provide the necessary flexibility with both high-level and more detail as appropriate.

[For details of the RBS concept and use, take a look at the paper where this idea was first described, available online at www.risk-doctor.com/pdf-files/rbs1002.pdf.]

The last blog entry addressed the need to distinguish risk from uncertainty. There are an infinite number of uncertainties, but these are only risks if they would affect objectives if they occurred. A risk is “an uncertainty that matters”.

Another common challenge in risk identification is to avoid confusion between causes of risk, genuine risks, and the effects of risks. The PMI^®  PMBoK^® Guide says that “A risk may have one or more causes and, if it occurs, one or more impacts”. In the most simple case, one cause leads to a single risk which in turn could have just one effect, though of course reality is considerably more complex. How do these three differ?

• Causes are definite events or sets of circumstances which exist in the project or its environment, and which give rise to uncertainty. Examples include the requirement to implement the project in a developing country, the need to use an unproven new technology, the lack of skilled personnel, or the fact that the organisation has never done a similar project before. Causes themselves are not uncertain since they are facts or requirements, so they are not the main focus of the risk management process.
• Risks are uncertainties which, if they occur, would affect the project objectives either negatively (threats) or positively (opportunities). Examples include the possibility that planned productivity targets might not be met, interest or exchange rates might fluctuate, the chance that client expectations may be misunderstood, or whether a contractor might deliver earlier than planned. These uncertainties should be managed proactively through the risk management process.
• Effects are unplanned variations from project objectives, either positive or negative, which would arise as a result of risks occurring. Examples include being early for a milestone, exceeding the authorised budget, or failing to meet contractually agreed performance targets. Effects are contingent events, unplanned potential future variations which will not occur unless risks happen. As effects do not yet exist, and indeed they may never exist, they cannot be managed directly through the risk management process.

Including causes or effects in the list of identified risks obscures genuine risks, which may not receive the appropriate degree of attention they deserve. So how can we clearly separate risks from their causes and effects? One way is to use risk metalanguage (a formal description with required elements) to provide a three-part structured “risk statement”, as follows : “As a result of <one or more definite causes>, <uncertain event or condition> may occur, which would lead to <one or more effects on objective(s)>.”

Examples include the following :

• “As a result of using novel hardware(a definite requirement), unexpected system integration errors may occur (an uncertain risk), which would lead to overspend on the project (a negative effect on the budget objective).”  
• “Because our organisation has never done a project like this before (fact = cause), we might misunderstand the customer's requirement (uncertainty = risk), and our solution would not meet the performance criteria (contingent possibility = effect on objective).”
• “We have to outsource production (cause); we may be able to learn new practices from our selected partner (risk), leading to increased productivity and profitability (effect).”

The use of risk metalanguage should ensure that risk identification actually identifies risks, distinct from causes or effects. Without this discipline, risk identification can produce a mixed list containing risks and non-risks, leading to confusion and distraction later in the risk process.