Who is Jerry Madden?

During Jerry Madden's 37-year career at NASA, the federal agency launched its first satellite, achieved the first lunar landing, and deployed the Hubble telescope. It also innovated outside the edges, bringing satellite TV, air-cushioned sneakers, and solar panels to the masses. In other words, NASA was an idea factory running at full steam.

Madden, who retired in 1995 as associate director of flight projects at Goddard Space Flight Center, was critical to the operation. As one of NASA's premiere project managers, he saw to it that great ideas became tangible innovations; he coordinated the technology, teams, and bureaucracy needed to propel science forward.

Along the way, Madden also curated and penned a now-infamous list of 128 lessons for project managers, which still circulates through NASA today.

Source of this document

You can download the original (free) at http://go.nasa.gov/2fBULlK  But some of it is NASA-specific or at least Aerospace-specific.   I’ve modified these slightly to make them less “application specific” and more in-tune with current Project Management theory.  I’m taking them 25 at a time and below are the first (edited) 25

From the original document: “None of these are original--It's just that we don't know where they were stolen from!”

The same goes for me!

Discussions:

I think the community here can add / subtract and modified from these.  Please feel free to post corrections, insults, additions, or general impressions.  Maybe even pick out your favorites. 

The Project Manager

1. There are no such thing as previously-fielded systems. The people who build the next unit probably never saw the previous unit; there are probably minor changes; the operational environment has probably changed; and the people who check the unit out will in most cases not understand the unit or the tests.


2. Most equipment works "as built," –  not as the designer planned. This is due to layout of the design, poor understanding on the designer's part, or poor understanding of component specifications.


3. The source of most problems is people but damned if they will admit it.  Know the people working on your project, so you know what the real weak spots are.


4. Most Project Managers succeed on the strength and skill of the project team


5. A manager who is his own systems engineer or financial manager is one who will probably try to do open heart surgery on themselves.


6. One must pay attention to workaholics – if they get going in the wrong direction, they can do a lot of damage in a short time – it is possible to overload them, causing premature burnout, but hard to determine if the load is too much, since much of it is self-generated.  It is important to make sure such people take enough time off and that the workload does not exceed 1-1/4 to 1-1/2 times what is normal.


7. NASA projects compete for budget funds--they do not compete with each other. So, never attack any other project with the idea you should get their funding. Sell what you have on its own merit.

8. Suppliers respond well to the customer who pays attention to what they are doing, but not too well to the customer that continually second-guesses their activity. The basic rule is: a customer is always right, but the cost will escalate if a customer always has things done his way, instead of the way the supplier had planned. The ground rule is never change a supplier's plans unless they are flawed or too costly.  Remember the old saying, "better is the enemy of good."


9. Never undercut the project team in public.  Don't make decisions on work that you have given them to do in public meetings. Even if you direct a change, never take the responsibility for implementing away from the project team


10. The project has many resources. This is a powerful resource that can be used to attack problems.


11. Know who the decision makers on the project are. It may be someone outside of the project team who has the ear of executive management or someone in the chain of command. Whoever they are, try to get a line of communication to them on a formal or informal basis.


12. The program and project manager should work as a team. The program manager is your advocate at HQ and must be tied in to the decision making and should aid your efforts to be tied in too.


13. A project manager should visit everyone who is building anything for their project. People like to know that the project manager is interested in their work, and the best proof is for the manager to visit them and see first-hand what they are doing.


14. Never ask your management to make a decision that you can make. Assume you have the authority to make decisions unless you know there is a document that states unequivocally that you cannot.

15. Wrong decisions made early can be salvaged, but "right" decisions made late cannot.


16. Never make excuses; instead, present plans of actions to be taken.


17. Never try to get even for some slight by another project. It is not good form--it puts you on the same level as the other person--and often ends up hindering the project getting done.


18. If you cultivate too much egotism, you may find it difficult to change your position- -especially if the project team tells you that you are wrong. You should instill an attitude on the project whereby the team know they can tell you of wrong decisions.


19. One of the advantages of NASA in the early days was the fact that everyone knew that the facts we were absolutely sure of –  could be wrong.


20. Project Managers who rely on the paperwork to do the reporting of activities are known failures.


21. Not all successful Project Managers are competent and not all failed Project Managers are incompetent. Luck still plays a part in success or failure, but luck favors the competent, hard-working Project manager.


22. If you have a problem that requires the addition of people to solve, you should approach recruiting people like a cook who has under-salted, i.e., a little at a time.


23. A project manager must know what motivates the project suppliers, i.e., their award system, their fiscal system, their policies, and their company culture.


24. If there isn’t secret information on the project--so don't treat anything like it is secret. Everyone does better if they can see the whole picture, so don't hide any of it from anyone.


25. Know the resources at your location and if possible other locations. Other locations, if they have the resources, are normally happy to help. It is always surprising how much good help one can get by just asking. 

NASA Virtual PM Challenge

Did you know that NASA has publicly available Project Management Podcasts, presentations and a NASA Virtual PM Challenge?  This is all available at: http://go.nasa.gov/2ft5X4f  The sessions are available to the public.  Each session is live, interactive, with opportunity for the audience to pose questions to the speaker via a session Moderator.  Sessions are recorded and made available for on-demand viewing.   

The VISION of APPEL is “Through its world-class training curriculum, development programs, and strategic communications, the Academy helps ensure NASA’s project management and systems engineering communities have the skills and knowledge they need to advance mission success.”

But wait, there’s more!  They’re a PMI REP!  his means the Academy’s project management training counts toward the requirements for PMI®‘s Continuing Certification Requirements Program. Participants who attend registered PMI-approved Academy courses can earn Professional Development Units (PDUs).  They’re also closely associated with The International Council on Systems Engineering (INCOSE), The International Program/Project Management Committee (IPMC)

2016 Session 2: Project Derailed? Get Back on Track with Schedule and Cost

Abstract: When they agree to lead an aerospace project, project managers commit to delivering a product that is technically excellent within a specified schedule and budget. But what can they do if a project falls behind schedule or runs over budget?  

In this second session of the NASA’s Virtual PM Challenge series on cost, schedule, and risk, we will look at the actions project managers can take to recover in-house instrument projects that are exceeding budget or behind schedule. Specifically, we’ll examine the project manager-business manager partnership and how a high-functioning partnership translates into project success.

Speakers will be Kate Earle, Chief Learning Officer of the Quiet Leadership Institute, Jason Lee, Assistant Director for the Applied Research and Methods team at GAO, Vernell Jackson, Engineering Systems Resource Manager of Goddard Space Flight Center’s (GSFC) Applied Engineering & Technology Directorate, and Cynthia Simmons, Associate Division Chief of GSFC’s Instrument Systems and Technology Division. Moderator Ramien Pierre is from NASA’s Academy for Program Project and Engineering Leadership (APPEL).

Here’s some information on the last session:

2016 Session 3: Considering It All For Project Success: Dissenting Opinions at NASA

Abstract: Program and project managers must make thousands of decisions in the course of delivering successful products and missions. But how can they be certain their decisions are based on unvarnished inputs from their team members? The final session of NASA’s Virtual Project Management (PM) Challenge three-part series on schedule, cost, and risk will look at how project and program managers might reduce project risk by actively encouraging the expression of dissenting opinions among their team members.

Speakers will be Nigel Packham, Manager of the Flight Safety Office at Johnson Space Center (JSC), and Peter Spidaliere, Mission Systems Engineer at Goddard Space Flight Center (GSFC). Moderator will be Ramien Pierre from NASA’s Academy for Program/Project and Engineering Leadership (APPEL).

Here are a few interesting slides from the DOWNLOADABLE PowerPoint Deck

Place yourself back in front of the Standing Review Board – you MUST address the final of the six required judgement criteria for your project.  There are, of course, many other items you’ll need to address, but this is the last of the minimal set.

THE GOAL OF RISK MANAGEMENT

• To manage risk in a holistic and coherent manner across the Agency
  • Agency strategic goals explicitly drive risk management activities at all levels
  • All risk types and their interactions are considered collectively during decision-making
  • Implementation of risk management in the context of complex institutional relationships (programs, projects, centers, contractors, …)
• To better match the stakeholder expectations and the “true” resources required to address the risks to achieve those expectations
  • Better comprehension of the risk that a Better comprehension of the risk that a decision decision-maker is accepting when is accepting when making commitments to stakeholders
  • Having an integrated perspective of risks when analyzing competing alternatives
• To better establish close ties between the selected alternative and the requirements derived from it
  • Derivation of achievable requirements through systematic characterization of uncertainties

ADEQUACY OF THE RISK MANAGEMENT APPROACH

NASA takes risk management VERY seriously.  In this blog, I’ve reduced the scope and detail of the complete NASA risk management approach to be applicable to a wide range of different industries and applications.  NASA’s Risk Management program provides a unified structure that applies to all agency activities to ensure that risk management decisions are delegated and/or elevated to the appropriate level.   The full Risk-Informed-Decision-Making handbook is 128 pages long, and that’s just one of the references I’m using.  My goal is to give the reader a “taste” of what each PM must know about risk management – a lot!

Risk Management includes opportunity management — recognizing that spaceflight is an inherently risky endeavor and that the proper attitude towards risk management is to reach an optimal balance between minimizing the potential for loss while maximizing the potential for gain (opportunity).

All forms of Risk Management consist of two main and joined processes:

1. A Risk (and opportunity) Informed Decision Making (RIDM) addresses informed selection of decision alternatives to assure effective approaches to achieving objectives.
2. Continuous Risk (and Opportunity) Management (CRM) addresses implementation of the selected alternative to assure that requirements are met.

The Project Manager is required to be totally conversant on the adequacy of their project’s risk management approach including:

• Risk-management plans and processes including:
  • Risk Informed Decision Making (RIDM)
  • Continuous Risk Management (CRM)
• Open and accepted risks
• Risk assessments
• Risk mitigation plans
• Resources for managing/mitigating risks.

NASA’s Definition of Risk

The definition of risk used is very like what is described in the PMBOK® guide as an output of “Identify Risks” and placed in the risk register (PMBOK® 11.2.3.1).  A risk is defined by: “EVENT may occur causing IMPACT, or If CAUSE exists, EVENT may occur leading to EFFECT.”

NASA defines this as a “Triplet”

• The scenario(s) leading to degraded performance with respect to one or more performance measures
• The likelihood(s) (qualitative or quantitative) of those scenarios.
• The consequences that would result if those scenarios were to occur.

Also, in agreement with the PMBOK® guide, the purpose of this type of risk definition is to be able to “sift” the high-probability, low-consequence risks from the low-probability, high-consequence risks.

THE NASA RISK MANAGEMENT PROCESS

NASA Risk Management processes are based on both Continuous Risk Management (CRM), which stresses the management of risk during implementation - and - Risk-Informed Decision Making (RIDM) which is concerned with analysis of important or direction-setting decisions.

Continuous Risk Management (CRM)

1 – Identify:  Search for and locate risks before they become problems or opportunities.  This is the process of transforming uncertainties and issues about a project into distinct (tangible) risks that can be described and measured.

2 – Analyze: Converts risk data into decision-making information. The process of examining the risks in detail to determine the extent of the risks, how they relate to each other, and which ones are the most important

3 – Plan: Translates risk information into decisions and mitigating (or enhancing) actions.  This part of the process deals with deciding what, if anything, should be done about a risk or set of related risks

4 – Track:   Answers the questions:  Are the risk indicators and actions plan followed?  This is the process in which risk status data are acquired, compiled, and reported

5 – Control: To make informed, timely, and effective decisions regarding risks and their mitigation or enhancement plans.  During this process the project team examines the tracking status reports for identified project risks and decides what actions to take based on the reported data

6 - Communicate & Document:  Provides information and feedback to the project on the risk activities, current risks, and emerging risks.   It is this process in which risk information is conveyed between all project stakeholders.

Risk Informed Decision Making (RIDM)

RIDM helps ensure that decisions between alternatives are conducted with an awareness of the risks associated with each.  This is done to help prevent late design changes which are often drivers of risk, cost overruns, schedule delays, and even cancellation.  Also, it has been found that most project cost-saving opportunities occur in the definition, planning, and early design phases of a project.

The RIDM process attempts to respond to some of the primary issues that have derailed programs in the past:

1. The mismatch between stakeholder expectations and the resources required to address the risks to achieve their expectations
2. The miscomprehension of the risk that a decision-maker is accepting when making commitments to stakeholders
3. The miscommunication in considering the respective risks associated with competing alternatives

The RIDM process acknowledges the role that human judgment plays in decisions, and that technical information cannot be the sole basis for decision making. This is not only because of inevitable gaps in the technical information, but also because decision making is an inherently subjective, values-based enterprise. 

RIDM is typically appropriate for decisions that have one or more of the following characteristics:

• High Stakes — High stakes are involved in the decision, such as significant costs, significant potential safety impacts, or the importance of meeting the project objectives.
• Complexity — The actual ramifications of alternatives are difficult to understand without detailed analysis.
• Uncertainty — Uncertainty in key inputs creates substantial uncertainty in the outcome of the decision alternatives and points to risks that may need to be managed.
• Multiple Attributes — Greater numbers of attributes cause a greater need for formal analysis.
• Diversity of Stakeholders — Extra attention is warranted to clarify objectives performance measures when the set of stakeholders reflects a diversity of values, preferences, and perspectives.

Throughout the RIDM process, interactions take place between the stakeholders, the risk analyst, the subject matter experts (SMEs), the Technical Authorities, and the decision-maker to ensure that the knowledge is properly integrated and communicated into the deliberations that inform the decision.

The RIDM Process

You can download a free copy of the RIDM process handbook at:  http://ow.ly/TCWH306qAq9

Part 1: Identification of Alternatives

Objectives are decomposed into an individual issue that is significant to some or all the stakeholders. In general, a performance measure has a “direction of goodness” that indicates the direction of increasingly beneficial performance measure values.

Considered are:

• Safety (e.g., avoidance of injury, fatality, or destruction of key assets)
• Technical (e.g., thrust or output, amount of observational data acquired)
• Cost (e.g., execution within allocated cost)
• Schedule (e.g., meeting milestones)

Part 2: Risk Analysis of Alternatives

In Risk Analysis of Alternatives, the performance measures of each alternative are quantified. 

It is incumbent on risk analyst to model each significant possible outcome, accounting for its probability of occurrence, in terms of the scenarios that produce it.  This produces a distribution of outcomes for each alternative, as characterized by probability density functions over the performance measures.  The depth of analysis needs to agree with the stakes and complexity of the decision situations being addressed.

Avoiding Decision Traps During Analysis

• Anchoring —The tendency of decision-makers to give extra weight to the first information they receive. It is related to a tendency for people to reason in terms of changes from a “baseline” and to formulate that baseline quickly and sometimes baselessly.

• Status Quo Bias — There is a tendency to want to preserve the status quo in weighing decision alternatives. Early designs of “horseless carriages” were strongly based on horse-drawn buggies, despite being not-optimal for engine-powered vehicles. There is also the tendency for managers to believe that if things go wrong with a decision, they are more likely to be punished for having acted vs. having allowed the status quo to continue.
• Sunk-Cost — The tendency to throw good money after bad: to try to recoup losses by continuing a course of action, even when the rational decision would be to walk away, based on the current state of knowledge. This bias is seen to operate in the perpetuation of projects that are floundering, to the point where additional investment diverts resources that would be better spent elsewhere.
• Confirmation Bias — The tendency to give greater weight to evidence that confirms our prior views.
• Framing — A class of biases that relate to the human tendency to respond to how a question is framed, regardless of the objective content of the question.  
• Overconfidence — The widespread tendency to underestimate the uncertainty that is inherent in the current state of knowledge. While most “experts” will acknowledge the presence of uncertainty in their assessments, they tend to do a poor job of estimating confidence intervals. This is particularly true for decisions that are challenging to implement, as many decisions at NASA are. In the face of multiple sources of uncertainty, people tend to pay attention to the few with which they have the most experience, and neglect others. It is also particularly true for highly unlikely events, where there is limited data available to inform expert judgment.
• Recallability — The tendency of people to be strongly influenced by experiences or events that are easier for them to recall, even if an analysis of that experience would yield a different answer. This means that dramatic or extreme events may play an unwarrantedly large role in decision making based on experience.

Part 3, Risk-Informed Alternative Selection

There are several approaches to selecting an alternative.  Deliberation takes place among the stakeholders and the decision-maker, and the decision-maker either culls the set of alternatives and asks for further scrutiny of the remaining alternatives OR selects an alternative for implementation OR asks for new alternatives.

Deliberation and decision making might take place in several venues over time.  The rationale for the  

• The risk deemed acceptable for each performance measure;
• The risk information and Risk Analysis of an Alternative Uncertain Conditions Performance Measure 1 Performance Measure n Probabilistically - Determined Outcomes Funding Environment Technology Development Limited Data Operating Environment Etc.
• Performance measures depicted for a single alternative Design, Test & Production Processes
  • Safety Risk …
  • Technical Risk
  • Cost Risk
  • Schedule Risk
• The pros and cons of each contending decision alternative, as discussed during the deliberations.

References

The following items are referenced in the text of this document:

1. NASA/SP-2010-576 – NASA Risk-informed Decision Making Handbook
2. Continuous Risk Management Guidebook – Software Engineering Institute, Carnegie Mellon University, 1996.
3. NPR 7120.5 – NASA Procedural Requirement, NASA Program and Project Management Processes and Requirements
4. NPR 8000.4 - NASA Procedural Requirement, Risk Management Procedural Requirements

The Fifth of Six Criteria That Each NASA Project Manager Must Know

5A: ADEQUACY AND AVAILABILITY OF RESOURCES OTHER THAN BUDGET

This is often examined during the Project Reviews with the simple question: “Do you have a handle on the resources you’ll need?”  Without the correct resources, available at the correct time, a project will certainly encounter trouble or even fail.  This critical need is examined by the review committee, often with recommendations being made. By now you are probably asking:  What happened to the budget?  Isn’t money a concern?  That was part of an earlier blog that discussed the Joint Cost and Schedule Confidence Level (JCL) This is the probability that cost will be equal to or less than the targeted cost, and schedule will be equal to or less than the targeted schedule date.  During the conduct of a project review – very little is missed! 

“Project teams should embrace the external reviews. External reviews allow the project to think about all the tough questions they’re going to be asked and give them time to plug the holes. Brainstorm possible questions with the team to make sure they are covered. Next, projects should determine what the decision points are and what decision trees should be used for addressing these points. The project manager should assign actions and revisit them prior to the review, using this as a preparation for the review.”

— JPL Programs and Projects Manager

Of course, the resources required for a project of course include more than people.  I’ve made this mistake a few times, and believe me, I hope I won’t do it again.  NASA projects require not only highly skilled people, but Commercial-Off-The-Shelf (COTS) components and perhaps unique, complex, one-of-a-kind items created by a vendor, or internal NASA shops.   These are all resources that are needed to complete the project.  It’s part of your duty to make sure you understand what you’ll need, when you’ll need it and what to do if it’s not available.  So, each PM is judged on their having a documented understanding of the total resources required to complete the scope of their Project

Quoting the NASA standard: “Adequacy and availability of resources other than other than budget are essential elements of successful project functionality, implementation and operation. These resources include: workforce, fabrication, assembly, test facilities and equipment, test beds, ground support equipment, launch sites, communication networks, and mission operation centers. They can be either government or privately held resources.”

Each Project Manager must have a handle on all resources and where the need for that resource came from – in other words, what requirement is driving the need?  This in-depth and personal understanding includes the planning, projected or current availability of components and staffing, competency and stability of staffing, required infrastructure, and the industrial base/supplier chain requirements.  There’s a lot of investigation, deep-thinking and planning required to create the needed comfort-factor that “requirements other than budget” are completely understood.?

Essentially the review board is looking for:

• Planning, availability, competency and stability of staffing, infrastructure

• The industrial base/supplier chain requirements

• Planning, availability, competency and stability of staffing, and infrastructure requirements.

Resource Dashboard

The standard reporting system for the Review Board is a three-level metric scale, i.e., successful (green), partially successful (yellow), or unsuccessful (red). This is sometimes referred to as a stop-light assessment.

For judging Resource Adequacy:

                Successful: (Green Status): All key implementation facilities have been identified and are available to support near term (5-year) missions.  This includes the availability, competency and Stability of staffing, essential infrastructure and additional resources are adequate for remaining lifecycle phases.

                Partially Successful: (Yellow Status):  All key resources, may not be identified to support near term (5-year) missions, known resources may not be available when needed, external resource needs are notional. Preliminary staffing and essential infrastructure requirements have been identified and documented; preliminary sources have been identified.

                Unsuccessful: (Red Status): Needed resources and/or facilities are not identified, availability of either internal or external resources are unknown.  Staffing resource needs are clearly inadequate.

The Review Board’s assessment will consider not only the adequacy of the proposed and acquired resources, but also alternatives that might reduce cost or risk, or perhaps improve the performance of associated life-cycle activities. As with the other assessments, the Review Board must understand the margins and constraints for the project especially as it relates to current and planned workforce loading

“Clearly, having a good system philosophy and well-transmitted expectations makes a big difference in how they do their jobs.”

– Project Manager, JSC

Overall Resource Acquisition Strategy: As early as possible in planning, all project types begin to define theirs acquisition strategy. The Acquisition Strategy is the plan or approach for using NASA’s acquisition authorities to achieve the project’s mission.   This includes recommendations from make/buy analyses, the recommendations from vendor competition analyses, proposed partnerships and contributions, proposed infrastructure use and needs, budget, and other considerations.

This documented strategy addresses the project’s initial plans for obtaining the systems, research, services, construction, and supplies that it needs to fulfill its mission, including any known procurement(s); the availability of the industrial base capability and supply chain needed to design, develop, produce, and support the project and its planned projects; identifying risks associated with single source or critical suppliers; and attendant mitigation plans

5b: A Technology Development Plan

This plan should describe the technology assessment, development, management, and acquisition strategies needed to achieve the project’s objectives.  It describes how the project will assess its technology development requirements, including how the project will evaluate the feasibility, availability, readiness, cost, risk, and benefit of the new technologies. It describes how the project will identify opportunities for leveraging ongoing technology efforts, including technology developed on other NASA projects or programs, at other governmental agencies, or in industry.

The Technology Development Plan also identifies the supply chain needed to manufacture the technology and any costs and risks associated with the transition from development to the manufacturing and production phases.  To accomplish these rather in-depth and detailed goals, the Technology Development Plan typically:

• Describes the project’s strategy for ensuring that there are alternative development paths available if/when technologies do not mature as expected

• Describes how the project will remove technology gaps, including

  •  Maturation, validation, and insertion plans

  •  Performance measurement at quantifiable milestones

  •  Off-ramp decision gates (i.e., the point during development where the project assesses whether the technology is maturing adequately and, if not, decides to terminate continued technology development); and required

• Describes briefly how the project will ensure that all planned technology exchanges, contracts, and partnership agreements comply with all laws and regulations regarding export control and the transfer of sensitive and proprietary information

• Describes the project’s technology utilization and commercialization plan in accordance with the requirements of NPD 7500.2, NASA Innovative Partnerships Project and NPR 7500.1, NASA Technology Commercialization Process

• Describes how the project will transition technologies from the development stage to manufacturing, production, and insertion into the end system

• Identifies any potential costs and risks associated with the transition to manufacturing, production, and insertion; and documents appropriate mitigation plans for the identified risks.

A Technology Development Plan is  a big deal…

Here is a “Technology Development Plan Appendix” (PDF download @ http://go.nasa.gov/2fhx8CO)  published this year for the ‘Exoplanet Exploration Program’ by the Jet Propulsion Laboratory at Cal Tech.

THE FOURTH OF SIX CRITERIA THAT A NASA PROJECT MANAGER MUST KNOW.

This blog discusses the fourth of six project criteria every Project Manager is held responsible for – and must clearly define for their project(s).  I’m taking these one at a time now, since they get a bit into the more technical side of Project Management.

“You, as a project manager, are called on to make some key decisions, but you are also riding on the top of the 95 percent of the good decisions that were made by the people you delegated to. So, it is a team activity and how you treat and manage those people … makes the real difference.”

 – Human Research Facility Project Manager, JSC

You need to imagine that we’re still doing a stand-up review in front of a LOT of smart people as described in my previous blogs.   People that have managed complex projects for years.   These are people that are experts in risk analysis as well as academics that understand the mission of the project also of course – your bosses are there.   This is a big deal!  It’s your opportunity to show how good a project manager you are – and – to learn things from the audience.  It’s a *very* interactive meeting.

Criteria 4. Adequacy of integrated cost and schedule estimate and funding strategy

I don’t want to get too nerdy here.  But the bottom line is that careful estimates are made, integrated with risk management and tracked with earned value. 

4A: Cost and schedule control plans

Yes, we’re talking about full-blown Earned Value Management!  (see https://evm.nasa.gov/)  Typically, effort uncertainty is modeled using a three-point estimate at the activity or a summary (Work Package) level. The lowest estimated value represents the low extreme of uncertainty, the middle value represents the “most likely” value of the cost or duration, and the high value represents the high extreme of uncertainty.  These estimates are linked to the identified risks for the project to establish a reasonable cost and perhaps schedule reserves.  

“Throughout the execution of the project, the Project Manager shall ensure that the results of all analysis based on EVM are linked to the Risk Management Plan of the Project. Any cost and/or schedule risks being managed by the Project Manger should rely on the results of EVM analysis to track, manage, and mitigate risks.” - NPR 9501.3

You can have reserves, but they must be smartly estimated, reviewed and disclosed.  “Undisclosed reserve” is a bad thing.  Reserves (both cost and schedule) can be handled different ways, but you MUST be consistent in the way they are managed and presented. Below are a few options that are offered as guidance.  

“The PM can make reserve numbers available within the project so all project team members know what reserves are, or the PM can keep reserve numbers quiet.  For example, a PM on a recent successful science mission allowed science instrument teams to have insight into how much reserve he had allocated for each instrument, but reserve was held at the project level. This openness allowed everyone to see the situation but also provided oversight and control” - NPR 9501.3

4B: Basis of Estimate (BoE)

A bottom-up (from the lowest level WBS elements) analysis is a common way to approach this.  Documenting the basis of estimate is often invaluable in the latter phases of the project.  The estimate’s focus should be on:

• Clarity of the objectives
• Thoroughness in the state of the technical and management plan
• Complexity of technology needs

This is exactly what is described in the PMBOK® Guide’s description of a Bottom-Up Estimate “Estimate Activity Resources” in paragraph 6.4.2.4

4C: Cost and schedule estimates consistent with project requirements assumptions, risks, and margins

Probably one of the most interesting item prepared and reviewed is the Joint Cost and Schedule Confidence Level (JCL) of the project.   This isn’t a PMBOK® Guide topic, but it certainly builds on what is addressed in the guide.

NASA’s Project Management requires that projects develop probabilistic risk-informed analyses of cost and schedule estimates to obtain a quantitative measure of the likelihood that the estimate will be met.  Risk analysis provides an analytical basis for establishing defensible cost estimates for likely project risks.  This analysis must be continuously reviewed and updated as more data become available.  A risk analysis, consists of answering the following questions:

• What can happen?
• How likely is it that it will happen?
• If it does happen, what are the consequences?

The cost analysis considers:

• Possible risks, threats, liens, uncertainties, mitigation strategies, and opportunities must be explicitly quantified, including the following:
  • Their probability of occurring
  • Their estimated cost and/or schedule consequences

Risk analysis utilizes modeling, analysis, and evaluation and contains various types of uncertainty. In general, these uncertainties may be attributable to several factors that include

• The statistical nature of data
• The insufficient understanding of physical and biological phenomena
• Unpredictable events (e.g., natural, biological, and human behavior).
• Impacts of cost and schedule performance to date

4D: Joint Cost and Schedule Confidence Level

Joint Cost and Schedule Confidence Level (JCL) is a process that combines a project’s cost, schedule, and risk into a complete picture.  The probability that the project cost will be equal to or less than the targeted cost and that schedule will be equal to or less than the targeted schedule date.   This helps inform management of the likelihood of a project’s success.

Why Do a JCL?

JCL analysis provides a cohesive and holistic picture of the project’s ability to achieve cost and schedule goals by integrating technical, cost, schedule, and risk data.  The project’s JCL can show the impacts of risk to a project as well as highlight the relationship between cost and schedule. This relationship can be extremely important in situations with constrained budgets. A complete JCL analysis also facilitates transparency with stakeholders on expectations and probabilities of meeting those expectations.

What is the official definition of JCL?

JCL is:

• The probability that cost will be equal or less than the targeted cost AND schedule will be equal or less then the targeted schedule date
• A process and product that helps inform management the likelihood of a projects’ programmatic success
• A process that combines a projects’ cost, schedule, and risk into a complete picture

JCL is not a:

• Specific methodology (e.g. resource loaded schedule)
• Product from a specific tool (e.g. @RISK)

The Four Key JCL Inputs:

1. DEVELOP A SUMMARY ANALYSIS SCHEDULE
   • Build a logic network of activities.
   • Utilizing a summary analysis schedule can significantly improve the process.

2. LOAD COST ONTO THE SCHEDULE ACTIVITIES
   • Map cost to the schedule.
   • Cost data can be summarized by a work breakdown structure (WBS) to aid with mapping.

3. INCORPORATE RISK LIST
   • Quantify likelihood of occurrence and impact.
   • Map risks to the appropriate activities.

4. CONDUCT UNCERTAINTY ANALYSIS
   • Apply uncertainty to the schedule and cost.  (The results of a quantitative analysis)

CALCULATE AND VIEW RESULTS

To calculate a JCL, the project combines its cost, schedule, and risk into a single model that can generate a probabilistic assessment of the level of confidence of achieving a specific cost-schedule goal.  Programs must be baselined at a 70 percent probability that the projects will be completed at or below the estimated cost and at or before the projected schedule.

The JCL scatterplot is a standard XY chart with schedule on the X-axis and cost on the Y-axis. Each point is a result of the simulation calculation representing one cost and schedule pair. The JCL calculation is based on the number of results in the target area for both cost and schedule and is expressed as a percentage of all simulation results displayed on the scatterplot.

Establishing a cost and schedule target (black dotted lines) divides the scatterplot into two areas. One area contains results that are at or beneath the target (shown in green). The other area contains results that exceed the target (shown in blue).

The yellow points and frontier line represent all results from the simulation that meet a desired Joint Confidence Level. Multiple points from the simulation may meet the JCL target. Each of the yellow points would establish a new target area large enough to meet the desired JCL

The Joint Cost and Schedule Confidence Level model describes the:

• Basis for base schedule duration and logic
• Basis for baseline cost estimates
• Risks included and basis for probability and consequences
• Risks excluded and why
• Description of the JCL method use

Analyze the scatterplot, run sensitivities, and refine!

Facts and Myths About JCL

MYTH: JCL analysis requires expensive software tools.

FACT: NASA has JCL tools available at no cost to the projects.

MYTH: A JCL requires a detailed resource-loaded schedule.

FACT: Completing a JCL requires only costs, not labor categories and rates.

MYTH: A JCL must be based on a detailed integrated master schedule (IMS).

FACT:  Summary and analysis schedules are preferred!

A full-blown Project Joint Cost and Schedule Confidence Level