Introduction
Developing reliable cost estimates is crucial for realistic program planning, budgeting and management. While some agency guidelines on cost estimating are thorough, other agency guidance is limited regarding processes, procedures, and practices for ensuring reliable cost estimates. The Cost Guide is intended to address that gap. Its purpose is twofold—to address generally accepted best practices for ensuring reliable cost estimates (applicable across government and industry) and to provide a detailed link between cost estimating and earned value management (EVM). Providing that link is especially critical, because it demonstrates how both elements are necessary for setting realistic baselines and managing risk. As a result, government managers and auditors can use the best practices in the Cost Guide to assist them as they assess (1) the reliability of a program’s cost estimate for budget and decision-making purposes, and (2) the program’s status using EVM.
The Cost Guide outlines key steps in the cost estimating process: the purpose, scope, and schedule of a cost estimate; a technical baseline description;3 a work breakdown structure (WBS); ground rules and assumptions; data collection; estimating methodologies; sensitivity and risk analysis; documenting and presenting results; and updating estimates with actual costs. The Guide also includes information on EVM; the composition of a competent cost estimating team; software cost estimating; and best practices for an analysis of alternatives. Additionally, the Guide addresses auditing and validating cost estimates. The Guide discusses pitfalls associated with cost estimating and EVM that can lead government agencies to accept unrealistic budget requests—such as when risks are unaccounted for in an otherwise logical approach to estimating costs.
Developing the Guide
Our approach to developing this Guide was to revise and update best practices and standard criteria originally published in GAO’s Cost Guide. To update the criteria for cost estimating standards, we consulted with a committee of cost estimating, scheduling, and earned value analysis specialists from across government, private industry, and academia. We sought input and feedback from all who expressed interest in revising the Cost Guide for three months. We describe our scope and methodology in detail in appendix I.
We intend to update the Cost Guide periodically. Comments and suggestions from experienced users are always welcome, as are recommendations from experts in the cost estimating and EVM disciplines.
The Cost Guide in Relation to Established Standards
Appendix XII provides information on how cost estimating standards relate to an entity’s internal control system. This Guide’s reference list identifies cost estimating guides and sources available from other government agencies and organizations that we relied on to determine the processes, practices, and procedures most commonly recommended in the cost estimating community. Users of the Guide may wish to refer to those references for more information. In addition, we relied on information from the International Cost Estimating and Analysis Association (ICEAA) and AACEI, which provide standards for cost estimating, and the Project Management Institute (PMI), which provides EVM standards.
The Guide’s Readers
The federal audit community is the primary audience for this Guide. Besides GAO, auditing agencies include Inspectors General and agency audit services. Additionally, agencies that do not have a formal policy for conducting or reviewing cost estimates will benefit from the Guide because it will inform them of the criteria GAO uses in assessing a cost estimate’s reliability. The National Science Foundation, Federal Railroad Administration, and Missile Defense Agency are examples of agencies that have aligned their cost estimating guidance to GAO’s cost estimating best practices.
The Guide’s Case Studies
The Cost Guide contains a number of case studies drawn from GAO program reviews. The case studies highlight problems typically associated with cost estimates and augment the key points and lessons learned that the chapters discuss. Appendix II gives some background information for each program used in the case studies. Some case studies in this Guide are reprinted from GAO reports that are several years old. These case studies are reflective of agency practices at that time and are provided for illustration purpose only.
Applicability of the Guide
Throughout this Guide, we refer primarily to cost estimates that encompass major system acquisitions, although the best practices in the Guide are equally applicable to capital and non-capital program cost estimates. Since its publication in 2009, we have applied the cost estimating best practices as auditing criteria to a myriad of capital and non-capital programs. These criteria are not limited to large-scale, non-real estate programs such as weapons systems, spacecraft, aircraft carriers, and software systems. We have applied the cost estimation practices in past work involving communications networks, the decennial census, high speed rail projects, and federal construction and maintenance projects. The practices are also applicable to government in-house development efforts for which a cost estimate must be developed to support a budget request. Wherever possible, the Guide uses generalized cost estimating terminology for practices and related documentation. However, in certain cases we use Department of Defense terminology because DOD is considered by many experts to be a leading agency in cost estimating techniques.
In this Guide, we use the term “program,” but some agencies may make distinctions between programs, projects, activities, functions, policies, or products. For the purposes of this Guide, these terms can be used interchangeably to accommodate an agency’s particular application. The processes and best practices developed in this Guide are intended for use in any acquisition, program, project, activity, function, policy, or product that benefits from the use of cost estimating and earned value management.
Finally, while we briefly discuss economic analyses in an overview of cost analysis in chapter 2, this Guide does not pertain to economic analyses.
The New Cost Guide
Chapters 1-16 of the Cost Guide discuss the importance of cost estimating and best practices associated with creating reliable cost estimates. They describe how cost estimates are used to predict, analyze, and evaluate a program’s cost and schedule and serve as a critical program control planning tool. The first two chapters discuss government’s need for cost estimating and a general overview of types of cost estimates and related analyses. Chapter 3 introduces the 12 steps in developing a reliable cost estimate, the associated best practices, and how the steps and best practices relate to the four characteristics of a reliable estimate. Chapters 4 through 15 address each of the 12 cost estimating steps. Chapter 16 is a new chapter focused on the auditor, and recaps steps, best practices, and characteristics and the process for auditing and validating an estimate.
Chapters 17-20 discuss cost management and the use of earned value management for measuring program performance against an approved baseline plan. Those chapters address best practices in implementing and integrating cost estimating, system development oversight, and risk management and their use to manage costs throughout the life of a program.
Changes from the 2009 Cost Guide
We have revised the initial version of the Cost Guide for the following reasons:
- To better describe the alignment of best practices, cost estimate characteristics, and cost estimating steps
- To clarify some of the best practices and their related criteria
- To provide additional content in technical appendixes and revise or delete others
- To update case studies and references to legislation and rules
- To modernize the Guide’s format and graphics
We developed one new chapter and significantly revised a second one to better describe the alignment of best practices to the characteristics of a reliable cost estimate and the process for creating reliable estimates. Chapter 3 introduces the four characteristics: comprehensive, well documented, accurate, and credible. It also introduces the 18 best practices and shows how the best practices align to the four characteristics. Additionally, it introduces the 12 steps of the cost estimating process that produce reliable estimates, and shows how the best practices align with the 12 steps. Chapter 16 reviews the characteristics and best practices in the context of auditing and validating the cost estimate. It describes each characteristic, its associated best practices, and includes effects that may occur if the best practices are not met. Chapter 16 also explains to the auditor how to assess a cost estimate, determine the reliability of a cost estimate, and how to evaluate an organization’s cost estimating guidance.
We added a survey of each cost estimating step at the end of its associated chapter. The survey describes the cost estimate process tasks and associated best practices. It also includes likely effects if the associated criteria are not fully met.
We have clarified some of the best practices. For example, the original Guide states that it is a best practice to present the estimate to management in a briefing. We recognize that organizations may use other methods to inform management about cost estimates and have updated the best practice by generalizing the requirement. In the updated Guide, any form of presentation to management is a valid means of meeting the best practice. Additionally, we previously described the technical baseline as a single document that includes detailed technical, program, and schedule information about a system. We have found that some programs describe this system information in a collection of documents. Thus, we have generalized our description of the technical baseline to be a single document or several documents stored in one location.
We have added two new appendixes, one that discusses best practices for analysis of alternatives (AOAs) and the second describing the relationship of internal controls to the cost estimating process. We have removed three appendixes. The appendix covering federal cost estimating and earned value legislation has been streamlined and is now included as a table in chapter 16. Because schedule risk analysis is covered in depth in GAO’s Schedule Assessment Guide, we have removed that appendix from the Guide. Additionally, we moved the software chapter to the appendix and deleted the previous appendix on software cost estimating risks. Finally, some material related to award fee criteria and progress payments was removed from chapter 18.
The original Cost Guide includes case studies from 1998-2007. Where possible, we have used GAO reports published since then to include more recent case studies within the Guide. Additionally, we have updated references in this Guide.
The original Cost Guide was published in 2009. Since then, GAO has updated and modernized the style and appearance of its best practice guides. This Guide has been revised to align with its companion guides, the Schedule Assessment Guide and the Technology Readiness Assessment Guide.
Acknowledgments
The Cost Guide team thanks the many members of the cost community who helped to develop and improve the Guide. After we discussed our plans for performing an update to the original Guide with members of the cost community, several experts expressed interest in working with us. Their contributions are invaluable.
Together with these experts, GAO has developed a Guide that clearly outlines its criteria for assessing cost estimates and EVM data during audits that we believe will benefit all agencies in the federal government. We would like to thank everyone who gave their time by attending meetings, giving us valuable documentation, and providing comments.
A technical baseline description is a document or set of documents that describe the program’s or project’s purpose, system, performance characteristics, and system configuration.↩︎