Conducting a Cost Risk and Uncertainty Analysis
To perform a risk and uncertainty analysis,33 WBS elements or risk drivers are assigned probability distributions of possible values. Statistical software randomly draws from each distribution and the cost results are summed for each iteration. This random drawing from distributions and total summation is repeated thousands of times. The resulting cumulative distribution curve displays the probability associated with the range of possible total program costs. Because the simulation’s inputs are probability distributions, the outputs are also distributions. The total cost distribution tends to be a lognormal distribution because many of the underlying distributions tend to be skewed to the right—that is, there is a greater probability of large overruns than large underruns. In setting up the simulation, any identified causality may be modeled.
The total cost distribution may be converted to a cumulative distribution function, or an S curve. An S curve is particularly useful for portraying the confidence level, or percentile, of a cost estimate. Figure 14 shows an example of a cumulative probability distribution with various estimate values mapped to their corresponding percentiles.
Figure 14: A Cumulative Distribution Function, or S Curve
Management can use an S curve to choose a defensible level of contingency. In figure 14, the S curve shows that the likelihood is about 40 percent that the final cost of the program will be $825,000 or less. Cost contingency is calculated by comparing the point estimate with that of the simulation result at a desired level of confidence. For example, a program manager planning for a confidence level at the 70th percentile would budget at $1,096,000, about $271,000 more than the point estimate. Likewise, the program has an equal chance of overrunning or underrunning its budget at $908,000, which is at the 50 percent confidence level and the median of the distribution. At this confidence level, the program would require $83,000 of contingency. Note that the mean, or average, of the distribution will usually be greater than the 50 percent confidence level because of the greater probability of overruns. The estimator should identify the cumulative probability associated with the point estimate and the estimate at management’s level of desired confidence.
Without a risk and uncertainty analysis, management cannot determine a defensible level of contingency that is necessary to cover increased costs resulting from unexpected design complexity, incomplete requirements, technology uncertainty, and other risks and uncertainty.
Alternative approaches to Monte Carlo simulation exist and are in use throughout the cost estimating community. For example, the Department of Defense and the National Aeronautics and Space Administration, Joint Agency Cost Schedule Risk and Uncertainty Handbook (Washington, D.C.: March 12, 2014) describes the scenario-based, method of moments, historical data references, and risk scoring methodologies. We discuss the 3-point and risk driver methodologies using Monte Carlo simulation because experts in the cost community consider them best practices.↩︎