Survey of Step 9

Process Tasks

• Conduct a risk and uncertainty analysis that includes the following steps:
  • Model probability distributions based on data availability, reliability, and variability.
  • Account for correlation between cost elements.
  • Use a Monte Carlo simulation model (or other modeling technique) to develop a distribution of total possible costs and an S-curve showing alternative cost estimate probabilities.
  • Identify the cumulative probability associated with the point estimate.
  • Identify contingency for achieving the desired confidence level.
  • Allocate the risk-adjusted cost estimate to WBS elements, if necessary.
  • Phase and convert the risk-adjusted estimate into budget year dollars.
  • Perform a risk and uncertainty analysis periodically as the cost estimate is updated to reflect progress and changes to risks.

Best Practices

A risk and uncertainty analysis is conducted that quantifies the imperfectly understood risks and identifies the effects of changing key cost driver assumptions and factors.

• Probability distributions are modeled based on data availability, reliability, and variability.
• Correlation between cost elements is captured.
• A Monte Carlo simulation model (or other modeling technique) is used to develop a distribution of total possible costs and an S curve showing alternative cost estimate probabilities.
• The cumulative probability associated with the point estimate is identified.
• Contingency is identified for achieving the desired confidence level.
• The risk-adjusted cost estimate is allocated to WBS elements, as necessary.
• The risk-adjusted cost estimate is phased and converted to budget year dollars.
• A risk and uncertainty analysis is performed periodically as the cost estimate is updated to reflect progress and changes to risks

Likely Effects If Criteria Are Not Fully Met

• Without a risk and uncertainty analysis, the program estimate will not reflect the degree of uncertainty, and a level of confidence cannot be given about the estimate. Unless a range of costs is provided, decision-makers will lack information on cost, schedule, and technical risks, and will not have insight into the likelihood of executing the program within the cost estimate.
• Lacking risk and uncertainty analysis, management cannot determine a defensible level of contingency that is necessary to cover increased costs resulting from unexpected design complexity, incomplete requirements, technology uncertainty, and other uncertainties.
• If risks are not accounted for and analyzed, cost estimators may underestimate or overestimate program costs.
• Unless a risk and uncertainty analysis is conducted and a program’s potential range of costs is assessed, management will lack information on whether the program fits within the overall risk range of the agency’s portfolio.
• If the risk and uncertainty analysis has been poorly executed or is based upon low-quality data, management may get a false sense of security that all risks have been accounted for and that the analysis is based on sound data. When this happens, program decisions will be based on bad information.
• If cost estimators only focus on the risks that most concern the program office or contractor, rather than a broad range of potential risks, program decisions may be based on poor quality information.
• If correlation is ignored, the risk and uncertainty analysis will likely understate the spread of the probability distribution about the total cost, resulting in a false sense of confidence in the estimate.
• Without an S curve, decision-makers will lack insight of what the likelihoods of different funding alternatives imply about program success. Furthermore, management will be less likely to proactively monitor a program’s costs because it does not know the likelihood of incurring overruns.
• Without an understanding of which input variables have a significant effect on a program’s final costs, management cannot efficiently devote resources to acquire better knowledge about those inputs to respond to their risks.
• If the risk and uncertainty analysis is not updated periodically, the following cannot be determined: the likelihood of completing the program within budget, the amount of contingency needed to provide an acceptable level of confidence in the required budget, and the risks most likely to impact the program cost.