Project Future Performance

Identify the Work That Remains

Two things are needed to project future performance: the actual costs spent on completed work, and the expected cost of remaining work. Actual costs spent on completed work are captured by the ACWP. The remaining work is determined by subtracting BCWP from BAC to calculate the budgeted cost of work remaining. To more accurately estimate the cost of remaining work, the EAC should take into account performance to date.

Calculate a Range of EACs and Compare to Available Funding

EVM data can be used to develop a multitude of EACs, and it is a best practice to develop more than one EAC. By calculating a range of EACs, management can know a likely range of costs for completing the program and take action in response to the results. However, picking the right EAC is challenging because the perception is that bad news about a contract’s performance could put a program and its management in jeopardy.

While plenty of EACs can be generated from the EVM data, each EAC is calculated with a generic index-based formula similar to:

EAC = ACWP (cumulative) + (BAC - BCWP (cumulative)) / efficiency index

The difference in EACs is driven by the efficiency index that is used to adjust the remaining work according to the program’s past cost and schedule performance. The efficiency index incorporates the concept that how a program has performed in the past will indicate how it will perform in the future. The typical performance indexes include the CPI and SPI (defined in table 24), but these could represent cumulative, current, or average values over time. In addition, the indexes could be combined to form a schedule cost index—as in CPI x SPI—which can be weighted to emphasize either cost or schedule impact. Further, EACs can be generated with regression analysis in which the dependent variable is ACWP and the independent value is BCWP, a performance index, or time. Thus, many combinations of efficiency indexes can be applied to adjust the cost of remaining work. Table 24 summarizes findings from studies describing which EACs make the best predictors, depending on where the program is in relation to its completion.

Table 24: Best Predictive Estimate at Completion (EAC) Efficiency Factors by Program Completion Status

Scroll to the right to view full table.

		Percent Complete
EAC Efficiency Factor		Early: 0%-40%	Middle: 20%-80%	Late: 60%-100%	Comment
Cost Performance Index (CPI)	Cumulative	x	x	x	Assumes the contractor will operate at the same efficiency for remainder of program; typically forecasts the lowest possible EAC
	3-month average	x	x	x	Weights current performance more heavily than cumulative past performance
	6-month average		x	x	Weights current performance more heavily than cumulative past performance
	12-month average		x	x	Weights current performance more heavily than cumulative past performance
CPI x Schedule Performance Index (SPI)	Cumulative	x	x		Usually produces the highest EAC
	6-month average		x	x	A variation of this formula (CPI6 x SPI), also proven accurate
SPI	Cumulative	x			Assumes schedule will affect cost also but is more accurate early in the program than later
Regression		x			Using CPI that decreases within 10 percent of its stable value can be a good predictor of final costs
Weighted		x		x	Weights cost and schedule based on .x(CPI) + .x(SPI); statistically the most accurate, especially when using 50 percent CPI x 50 percent SPI

Source: DOD. | GAO-20-195G

The findings in table 24 are based on extensive research that compared efficiency factors that appeared to best predict program costs. The conclusion was that no single factor was superior. Instead, the best EAC efficiency factor changes by the stage of the program. For example, the research found that assigning a greater weight to SPI is appropriate for predicting costs in the early stage of a program but not later in program development. SPI loses its predictive value as a program progresses and eventually returns to 1.0 when the program is complete. The research also found that averaging performance over a shorter period of time—3 months, for example—was more accurate for predicting costs than longer periods of time—such as 6 to 12 months—especially in the middle of a program when costs are being spent at a greater rate.

Other methods, such as the Rayleigh model, rely on patterns of manpower build-up and phase-out to predict final cumulative cost. This model uses a nonlinear regression analysis of ACWP against time to predict final cumulative cost and duration, and has been known to yield a high EAC forecast. One benefit of using this model is that as long as actual costs are available, they can be used to forecast cumulative cost at completion and to assess overall cost and schedule risk.

Relying on the CPI and SPI performance factors usually results in higher EACs if their values are less than 1.0. How much the cost will increase depends on the specific index and how many months are included in determining the factor. Research has shown that once a program is 20 percent complete, the cumulative CPI does not vary much from its value (less than 10 percent) and most often tends to get worse as completion grows nearer. Therefore, projecting an EAC by using the cumulative CPI efficiency factor tends to generate a best-case EAC.

In contrast, the schedule cost index—some form of CPI x SPI—takes the schedule into account to forecast future costs. This index produces an even higher EAC by compounding the effect of the program’s being behind schedule and over cost. The theory behind this index is that to get back on schedule will require more money because the contractor will either have to hire more labor or pay for overtime. As a result, the schedule cost index forecast is often referred to as a worst-case predictor.

A more sophisticated EAC method relies on summing the actual costs to date, the remaining work with a cost growth factor applied, and a cost impact for probable schedule delays. This EAC method also considers risks from the program risk register that may impact remaining cost and schedule, such as test failures or other external factors that have occurred in other past programs. This method relies on simulation to determine the probability effect.

Finally, an integrated schedule can be used in combination with risk analysis data and Monte Carlo simulation software to estimate schedule risk and the EAC.

EACs should be created not only at the program level but also at lower levels of the WBS. By doing so, areas that are performing poorly will not be masked by other areas doing well. If the areas performing worse represent a large part of the BAC, then this method will generate a higher and more realistic EAC.

Once a range of EACs has been developed, the results should be analyzed to determine if additional funding is required. Independent EACs provide a credible rationale for requesting additional funds to complete the program, if necessary. Their information is critical for better program planning and avoiding a situation in which work must be stopped because funds have been exhausted. Early warning of impending funding issues enables management to take corrective action to avoid any surprises.

While EVM data are useful for predicting EACs, the contractor should also look at other performance information to develop an EAC. In particular, the contractor should:

• evaluate its performance on completed work and compare it to the remaining budget,

• assess commitment values for material needed to complete remaining work, and

• estimate future conditions.

This comprehensive, or bottom-up, EAC should periodically be developed using all information available to develop the best estimate possible. This estimate should also take into account an assessment of risk based on technical input from the team. Once the EAC is developed, it can be compared for realism against other EACs and historical performance indexes.

Determine Whether the Contractor’s EAC Is Feasible

Because a contractor typically uses methods outside EVM to develop an EAC, EVM and risk analysis results can be used to assess the EAC’s reliability. While the contractor’s EAC tends to account for special situations and circumstances that cannot be accurately captured by looking only at historical trends, it also tends to include optimistic views of the future.

As noted earlier, one way to assess the validity of the EAC is to compare the TCPI to the CPI. Because the TCPI represents the ratio of remaining work to remaining funding and indicates the level of performance the contractor must achieve and maintain to stay within funding goals, it can be a good benchmark for assessing whether the EAC is reasonable. Therefore, if the TCPI is greater than the CPI, this means that the contractor expects productivity to be higher in the future. To determine whether this is a reasonable assumption, analysts should look for supporting evidence that backs up this claim.

Looking again at the example of the airborne laser program discussed around figures 33-34, we see that while the contractor predicted no overrun at completion, there was a cumulative unfavorable cost variance of almost $300 million. According to this research statement, one could conclude that the program would overrun by $300 million or more. Using EVM data from the program, we predicted that the final overrun could be anywhere between $400 million and almost $1 billion by the time the program was done.

Calculate an Independent Date for Program Completion

Dollars can be reallocated to future control accounts by management, but time cannot. If a cost underrun occurs in one cost account, the excess budget can be transferred to a future account. But if a control account is 3 months ahead and another is 3 months behind, time cannot be shifted from the one account to the other to fix the schedule variance. Given this dynamic, the schedule variance should be examined in terms of the network schedule’s critical and near-critical paths to determine what specific activities are behind schedule. To project when a program will finish, management must know whether the activities that are contributing to a schedule variance are on the critical path or may ultimately be on that path if mitigation is not pursued. If they are, then any slip in the critical path activities will result in a slip in the program’s finish date; sufficient slippage in near-critical paths may ultimately have the same result. If the delayed activities will affect the program schedule, then an analysis, generally a schedule risk analysis, should be conducted to determine the most likely completion date. In addition, a schedule risk analysis should be conducted periodically to assess changes to the critical path and explain schedule reserve erosion and mitigation strategies for keeping the program on schedule.⁵⁶

Provide Analysis to Management

The ability to act quickly to resolve program problems depends on having information of their causes early. Management can make better decisions that lead to greater success if it has accurate progress assessments of program status. When problems are identified, they should be captured and managed within the program’s risk management process so that someone can be assigned responsibility for tracking and correcting them.

In addition, using information from the independent EACs and the contractor’s EAC, management should decide whether additional program funding should be requested and, if so, make a convincing case for more funds. When this happens, however, management should also be sure to link program outcomes to award-fee objectives.⁵⁷ For example, management can evaluate earlier CPRs to determine if they objectively depicted contract status and predicted certain problems. This approach supports performance-based reporting and rewards contractors for managing their contracts effectively and reporting actual conditions, reducing the need for additional oversight.

---

56. For a detailed explanation of schedule risk analysis performance, see GAO, Schedule Assessment Guide, GAO-16-89G (Washington, D.C.: Dec. 2015).↩︎

57. The purpose of award fee contracting is to provide motivation to the contractor for excellence in such areas as quality, timeliness, technical ingenuity, and cost effective management. It is important that award fee criteria be selected to properly motivate the contractor to perform well and encourage improved management processes during the award fee period. See the sidebar for more discussion about use of award fee.↩︎