Place yourself back in front of the Standing Review Board – you MUST address the final of the six required judgement criteria for your project.  There are, of course, many other items you’ll need to address, but this is the last of the minimal set.

THE GOAL OF RISK MANAGEMENT

• To manage risk in a holistic and coherent manner across the Agency
  • Agency strategic goals explicitly drive risk management activities at all levels
  • All risk types and their interactions are considered collectively during decision-making
  • Implementation of risk management in the context of complex institutional relationships (programs, projects, centers, contractors, …)
• To better match the stakeholder expectations and the “true” resources required to address the risks to achieve those expectations
  • Better comprehension of the risk that a Better comprehension of the risk that a decision decision-maker is accepting when is accepting when making commitments to stakeholders
  • Having an integrated perspective of risks when analyzing competing alternatives
• To better establish close ties between the selected alternative and the requirements derived from it
  • Derivation of achievable requirements through systematic characterization of uncertainties

ADEQUACY OF THE RISK MANAGEMENT APPROACH

NASA takes risk management VERY seriously.  In this blog, I’ve reduced the scope and detail of the complete NASA risk management approach to be applicable to a wide range of different industries and applications.  NASA’s Risk Management program provides a unified structure that applies to all agency activities to ensure that risk management decisions are delegated and/or elevated to the appropriate level.   The full Risk-Informed-Decision-Making handbook is 128 pages long, and that’s just one of the references I’m using.  My goal is to give the reader a “taste” of what each PM must know about risk management – a lot!

Risk Management includes opportunity management — recognizing that spaceflight is an inherently risky endeavor and that the proper attitude towards risk management is to reach an optimal balance between minimizing the potential for loss while maximizing the potential for gain (opportunity).

All forms of Risk Management consist of two main and joined processes:

1. A Risk (and opportunity) Informed Decision Making (RIDM) addresses informed selection of decision alternatives to assure effective approaches to achieving objectives.
2. Continuous Risk (and Opportunity) Management (CRM) addresses implementation of the selected alternative to assure that requirements are met.

The Project Manager is required to be totally conversant on the adequacy of their project’s risk management approach including:

• Risk-management plans and processes including:
  • Risk Informed Decision Making (RIDM)
  • Continuous Risk Management (CRM)
• Open and accepted risks
• Risk assessments
• Risk mitigation plans
• Resources for managing/mitigating risks.

NASA’s Definition of Risk

The definition of risk used is very like what is described in the PMBOK® guide as an output of “Identify Risks” and placed in the risk register (PMBOK® 11.2.3.1).  A risk is defined by: “EVENT may occur causing IMPACT, or If CAUSE exists, EVENT may occur leading to EFFECT.”

NASA defines this as a “Triplet”

• The scenario(s) leading to degraded performance with respect to one or more performance measures
• The likelihood(s) (qualitative or quantitative) of those scenarios.
• The consequences that would result if those scenarios were to occur.

Also, in agreement with the PMBOK® guide, the purpose of this type of risk definition is to be able to “sift” the high-probability, low-consequence risks from the low-probability, high-consequence risks.

THE NASA RISK MANAGEMENT PROCESS

NASA Risk Management processes are based on both Continuous Risk Management (CRM), which stresses the management of risk during implementation - and - Risk-Informed Decision Making (RIDM) which is concerned with analysis of important or direction-setting decisions.

Continuous Risk Management (CRM)

1 – Identify:  Search for and locate risks before they become problems or opportunities.  This is the process of transforming uncertainties and issues about a project into distinct (tangible) risks that can be described and measured.

2 – Analyze: Converts risk data into decision-making information. The process of examining the risks in detail to determine the extent of the risks, how they relate to each other, and which ones are the most important

3 – Plan: Translates risk information into decisions and mitigating (or enhancing) actions.  This part of the process deals with deciding what, if anything, should be done about a risk or set of related risks

4 – Track:   Answers the questions:  Are the risk indicators and actions plan followed?  This is the process in which risk status data are acquired, compiled, and reported

5 – Control: To make informed, timely, and effective decisions regarding risks and their mitigation or enhancement plans.  During this process the project team examines the tracking status reports for identified project risks and decides what actions to take based on the reported data

6 - Communicate & Document:  Provides information and feedback to the project on the risk activities, current risks, and emerging risks.   It is this process in which risk information is conveyed between all project stakeholders.

Risk Informed Decision Making (RIDM)

RIDM helps ensure that decisions between alternatives are conducted with an awareness of the risks associated with each.  This is done to help prevent late design changes which are often drivers of risk, cost overruns, schedule delays, and even cancellation.  Also, it has been found that most project cost-saving opportunities occur in the definition, planning, and early design phases of a project.

The RIDM process attempts to respond to some of the primary issues that have derailed programs in the past:

1. The mismatch between stakeholder expectations and the resources required to address the risks to achieve their expectations
2. The miscomprehension of the risk that a decision-maker is accepting when making commitments to stakeholders
3. The miscommunication in considering the respective risks associated with competing alternatives

The RIDM process acknowledges the role that human judgment plays in decisions, and that technical information cannot be the sole basis for decision making. This is not only because of inevitable gaps in the technical information, but also because decision making is an inherently subjective, values-based enterprise. 

RIDM is typically appropriate for decisions that have one or more of the following characteristics:

• High Stakes — High stakes are involved in the decision, such as significant costs, significant potential safety impacts, or the importance of meeting the project objectives.
• Complexity — The actual ramifications of alternatives are difficult to understand without detailed analysis.
• Uncertainty — Uncertainty in key inputs creates substantial uncertainty in the outcome of the decision alternatives and points to risks that may need to be managed.
• Multiple Attributes — Greater numbers of attributes cause a greater need for formal analysis.
• Diversity of Stakeholders — Extra attention is warranted to clarify objectives performance measures when the set of stakeholders reflects a diversity of values, preferences, and perspectives.

Throughout the RIDM process, interactions take place between the stakeholders, the risk analyst, the subject matter experts (SMEs), the Technical Authorities, and the decision-maker to ensure that the knowledge is properly integrated and communicated into the deliberations that inform the decision.

The RIDM Process

You can download a free copy of the RIDM process handbook at:  http://ow.ly/TCWH306qAq9

Part 1: Identification of Alternatives

Objectives are decomposed into an individual issue that is significant to some or all the stakeholders. In general, a performance measure has a “direction of goodness” that indicates the direction of increasingly beneficial performance measure values.

Considered are:

• Safety (e.g., avoidance of injury, fatality, or destruction of key assets)
• Technical (e.g., thrust or output, amount of observational data acquired)
• Cost (e.g., execution within allocated cost)
• Schedule (e.g., meeting milestones)

Part 2: Risk Analysis of Alternatives

In Risk Analysis of Alternatives, the performance measures of each alternative are quantified. 

It is incumbent on risk analyst to model each significant possible outcome, accounting for its probability of occurrence, in terms of the scenarios that produce it.  This produces a distribution of outcomes for each alternative, as characterized by probability density functions over the performance measures.  The depth of analysis needs to agree with the stakes and complexity of the decision situations being addressed.

Avoiding Decision Traps During Analysis

• Anchoring —The tendency of decision-makers to give extra weight to the first information they receive. It is related to a tendency for people to reason in terms of changes from a “baseline” and to formulate that baseline quickly and sometimes baselessly.

• Status Quo Bias — There is a tendency to want to preserve the status quo in weighing decision alternatives. Early designs of “horseless carriages” were strongly based on horse-drawn buggies, despite being not-optimal for engine-powered vehicles. There is also the tendency for managers to believe that if things go wrong with a decision, they are more likely to be punished for having acted vs. having allowed the status quo to continue.
• Sunk-Cost — The tendency to throw good money after bad: to try to recoup losses by continuing a course of action, even when the rational decision would be to walk away, based on the current state of knowledge. This bias is seen to operate in the perpetuation of projects that are floundering, to the point where additional investment diverts resources that would be better spent elsewhere.
• Confirmation Bias — The tendency to give greater weight to evidence that confirms our prior views.
• Framing — A class of biases that relate to the human tendency to respond to how a question is framed, regardless of the objective content of the question.  
• Overconfidence — The widespread tendency to underestimate the uncertainty that is inherent in the current state of knowledge. While most “experts” will acknowledge the presence of uncertainty in their assessments, they tend to do a poor job of estimating confidence intervals. This is particularly true for decisions that are challenging to implement, as many decisions at NASA are. In the face of multiple sources of uncertainty, people tend to pay attention to the few with which they have the most experience, and neglect others. It is also particularly true for highly unlikely events, where there is limited data available to inform expert judgment.
• Recallability — The tendency of people to be strongly influenced by experiences or events that are easier for them to recall, even if an analysis of that experience would yield a different answer. This means that dramatic or extreme events may play an unwarrantedly large role in decision making based on experience.

Part 3, Risk-Informed Alternative Selection

There are several approaches to selecting an alternative.  Deliberation takes place among the stakeholders and the decision-maker, and the decision-maker either culls the set of alternatives and asks for further scrutiny of the remaining alternatives OR selects an alternative for implementation OR asks for new alternatives.

Deliberation and decision making might take place in several venues over time.  The rationale for the  

• The risk deemed acceptable for each performance measure;
• The risk information and Risk Analysis of an Alternative Uncertain Conditions Performance Measure 1 Performance Measure n Probabilistically - Determined Outcomes Funding Environment Technology Development Limited Data Operating Environment Etc.
• Performance measures depicted for a single alternative Design, Test & Production Processes
  • Safety Risk …
  • Technical Risk
  • Cost Risk
  • Schedule Risk
• The pros and cons of each contending decision alternative, as discussed during the deliberations.

References

The following items are referenced in the text of this document:

1. NASA/SP-2010-576 – NASA Risk-informed Decision Making Handbook
2. Continuous Risk Management Guidebook – Software Engineering Institute, Carnegie Mellon University, 1996.
3. NPR 7120.5 – NASA Procedural Requirement, NASA Program and Project Management Processes and Requirements
4. NPR 8000.4 - NASA Procedural Requirement, Risk Management Procedural Requirements