Prepared to Launch: Growing up PM at NASA

THE FOURTH OF SIX CRITERIA THAT A NASA PROJECT MANAGER MUST KNOW.

This blog discusses the fourth of six project criteria every Project Manager is held responsible for – and must clearly define for their project(s).  I’m taking these one at a time now, since they get a bit into the more technical side of Project Management.

“You, as a project manager, are called on to make some key decisions, but you are also riding on the top of the 95 percent of the good decisions that were made by the people you delegated to. So, it is a team activity and how you treat and manage those people … makes the real difference.”

 – Human Research Facility Project Manager, JSC

You need to imagine that we’re still doing a stand-up review in front of a LOT of smart people as described in my previous blogs.   People that have managed complex projects for years.   These are people that are experts in risk analysis as well as academics that understand the mission of the project also of course – your bosses are there.   This is a big deal!  It’s your opportunity to show how good a project manager you are – and – to learn things from the audience.  It’s a *very* interactive meeting.

Criteria 4. Adequacy of integrated cost and schedule estimate and funding strategy

I don’t want to get too nerdy here.  But the bottom line is that careful estimates are made, integrated with risk management and tracked with earned value. 

4A: Cost and schedule control plans

Yes, we’re talking about full-blown Earned Value Management!  (see https://evm.nasa.gov/)  Typically, effort uncertainty is modeled using a three-point estimate at the activity or a summary (Work Package) level. The lowest estimated value represents the low extreme of uncertainty, the middle value represents the “most likely” value of the cost or duration, and the high value represents the high extreme of uncertainty.  These estimates are linked to the identified risks for the project to establish a reasonable cost and perhaps schedule reserves.  

“Throughout the execution of the project, the Project Manager shall ensure that the results of all analysis based on EVM are linked to the Risk Management Plan of the Project. Any cost and/or schedule risks being managed by the Project Manger should rely on the results of EVM analysis to track, manage, and mitigate risks.” - NPR 9501.3

You can have reserves, but they must be smartly estimated, reviewed and disclosed.  “Undisclosed reserve” is a bad thing.  Reserves (both cost and schedule) can be handled different ways, but you MUST be consistent in the way they are managed and presented. Below are a few options that are offered as guidance.  

“The PM can make reserve numbers available within the project so all project team members know what reserves are, or the PM can keep reserve numbers quiet.  For example, a PM on a recent successful science mission allowed science instrument teams to have insight into how much reserve he had allocated for each instrument, but reserve was held at the project level. This openness allowed everyone to see the situation but also provided oversight and control” - NPR 9501.3

4B: Basis of Estimate (BoE)

A bottom-up (from the lowest level WBS elements) analysis is a common way to approach this.  Documenting the basis of estimate is often invaluable in the latter phases of the project.  The estimate’s focus should be on:

• Clarity of the objectives
• Thoroughness in the state of the technical and management plan
• Complexity of technology needs

This is exactly what is described in the PMBOK® Guide’s description of a Bottom-Up Estimate “Estimate Activity Resources” in paragraph 6.4.2.4

4C: Cost and schedule estimates consistent with project requirements assumptions, risks, and margins

Probably one of the most interesting item prepared and reviewed is the Joint Cost and Schedule Confidence Level (JCL) of the project.   This isn’t a PMBOK® Guide topic, but it certainly builds on what is addressed in the guide.

NASA’s Project Management requires that projects develop probabilistic risk-informed analyses of cost and schedule estimates to obtain a quantitative measure of the likelihood that the estimate will be met.  Risk analysis provides an analytical basis for establishing defensible cost estimates for likely project risks.  This analysis must be continuously reviewed and updated as more data become available.  A risk analysis, consists of answering the following questions:

• What can happen?
• How likely is it that it will happen?
• If it does happen, what are the consequences?

The cost analysis considers:

• Possible risks, threats, liens, uncertainties, mitigation strategies, and opportunities must be explicitly quantified, including the following:
  • Their probability of occurring
  • Their estimated cost and/or schedule consequences

Risk analysis utilizes modeling, analysis, and evaluation and contains various types of uncertainty. In general, these uncertainties may be attributable to several factors that include

• The statistical nature of data
• The insufficient understanding of physical and biological phenomena
• Unpredictable events (e.g., natural, biological, and human behavior).
• Impacts of cost and schedule performance to date

4D: Joint Cost and Schedule Confidence Level

Joint Cost and Schedule Confidence Level (JCL) is a process that combines a project’s cost, schedule, and risk into a complete picture.  The probability that the project cost will be equal to or less than the targeted cost and that schedule will be equal to or less than the targeted schedule date.   This helps inform management of the likelihood of a project’s success.

Why Do a JCL?

JCL analysis provides a cohesive and holistic picture of the project’s ability to achieve cost and schedule goals by integrating technical, cost, schedule, and risk data.  The project’s JCL can show the impacts of risk to a project as well as highlight the relationship between cost and schedule. This relationship can be extremely important in situations with constrained budgets. A complete JCL analysis also facilitates transparency with stakeholders on expectations and probabilities of meeting those expectations.

What is the official definition of JCL?

JCL is:

• The probability that cost will be equal or less than the targeted cost AND schedule will be equal or less then the targeted schedule date
• A process and product that helps inform management the likelihood of a projects’ programmatic success
• A process that combines a projects’ cost, schedule, and risk into a complete picture

JCL is not a:

• Specific methodology (e.g. resource loaded schedule)
• Product from a specific tool (e.g. @RISK)

The Four Key JCL Inputs:

1. DEVELOP A SUMMARY ANALYSIS SCHEDULE
   • Build a logic network of activities.
   • Utilizing a summary analysis schedule can significantly improve the process.

2. LOAD COST ONTO THE SCHEDULE ACTIVITIES
   • Map cost to the schedule.
   • Cost data can be summarized by a work breakdown structure (WBS) to aid with mapping.

3. INCORPORATE RISK LIST
   • Quantify likelihood of occurrence and impact.
   • Map risks to the appropriate activities.

4. CONDUCT UNCERTAINTY ANALYSIS
   • Apply uncertainty to the schedule and cost.  (The results of a quantitative analysis)

CALCULATE AND VIEW RESULTS

To calculate a JCL, the project combines its cost, schedule, and risk into a single model that can generate a probabilistic assessment of the level of confidence of achieving a specific cost-schedule goal.  Programs must be baselined at a 70 percent probability that the projects will be completed at or below the estimated cost and at or before the projected schedule.

The JCL scatterplot is a standard XY chart with schedule on the X-axis and cost on the Y-axis. Each point is a result of the simulation calculation representing one cost and schedule pair. The JCL calculation is based on the number of results in the target area for both cost and schedule and is expressed as a percentage of all simulation results displayed on the scatterplot.

Establishing a cost and schedule target (black dotted lines) divides the scatterplot into two areas. One area contains results that are at or beneath the target (shown in green). The other area contains results that exceed the target (shown in blue).

The yellow points and frontier line represent all results from the simulation that meet a desired Joint Confidence Level. Multiple points from the simulation may meet the JCL target. Each of the yellow points would establish a new target area large enough to meet the desired JCL

The Joint Cost and Schedule Confidence Level model describes the:

• Basis for base schedule duration and logic
• Basis for baseline cost estimates
• Risks included and basis for probability and consequences
• Risks excluded and why
• Description of the JCL method use

Analyze the scatterplot, run sensitivities, and refine!

Facts and Myths About JCL

MYTH: JCL analysis requires expensive software tools.

FACT: NASA has JCL tools available at no cost to the projects.

MYTH: A JCL requires a detailed resource-loaded schedule.

FACT: Completing a JCL requires only costs, not labor categories and rates.

MYTH: A JCL must be based on a detailed integrated master schedule (IMS).

FACT:  Summary and analysis schedules are preferred!

A full-blown Project Joint Cost and Schedule Confidence Level