Identifying Factors for the Sensitivity Analysis

The first step of a sensitivity analysis requires analysts to identify the factors to be varied. The sources of variation should be well documented and traceable. Simply varying factors by a subjective plus or minus percentage is not useful and does not constitute a valid sensitivity analysis.

Uncertainty about the values of some, if not most, of the technical parameters is common early in a program’s design and development. Many assumptions made at the start of a program turn out to be inaccurate. Therefore, once the point estimate has been developed, it is important to determine how sensitive the total cost estimate is to changes in the factors. Some factors that are often varied in a sensitivity analysis are:

  • a shorter or longer life cycle;

  • the volume, mix, or pattern of workload;

  • potential requirements changes;

  • configuration changes in hardware, software, or facilities;

  • alternative assumptions about program operations, fielding strategy, inflation rate, technology heritage savings, and development time;

  • higher or lower learning curves;

  • changes in performance characteristics;

  • testing requirements;

  • acquisition strategy, whether multiyear procurement or dual sourcing, among others;

  • labor rates; and

  • growth in software size or amount of software reuse.

In a sensitivity analysis, the cost estimator should always include the factors that are most likely to change, such as an assumption that was made for lack of knowledge or one that is outside the program office’s control.

Another method for identifying parameters is to examine artifacts from related analyses, such as risk and uncertainty analysis. One such artifact is a tornado chart, a special type of bar chart that shows which parameters have the greatest effect—positive or negative—on the overall point estimate (figure 12 is an example).

Figure 12: Tornado Chart for a Sensitivity Analysis
Tip: Click the figure to view a larger version in a new browser tab.

Determining which parameters are key cost drivers can be done in several ways. One way to determine key cost drivers is to calculate the proportion of each cost element to the total cost. Those input variables affecting the highest proportion cost elements should be varied in a sensitivity analysis. However, analysts may want to consider the parameters contributing to high-risk cost elements as well, even if they are not cost drivers, because these elements may become schedule drivers.

Many factors that should be tested are determined by the assumptions and performance characteristics outlined in the technical baseline description and associated assumptions. There should be a clear link between the technical baseline parameters, assumptions, and the cost model input values examined by cost estimators in the sensitivity analysis.