Evidence
Requirements: Evidence
8.90
Auditors must obtain sufficient, appropriate evidence to provide a reasonable basis for addressing the audit objectives and supporting their findings and conclusions.
8.91
In assessing the appropriateness of evidence, auditors should assess whether the evidence is relevant, valid, and reliable.
8.92
In determining the sufficiency of evidence, auditors should determine whether enough appropriate evidence exists to address the audit objectives and support the findings and conclusions to the extent that would persuade a knowledgeable person that the findings are reasonable.
8.93
When auditors use information provided by officials of the audited entity as part of their evidence, they should determine what the officials of the audited entity or other auditors did to obtain assurance over the reliability of the information.
8.94
Auditors should evaluate the objectivity, credibility, and reliability of testimonial evidence.
Application Guidance: Evidence
8.95
Audit objectives may vary widely, as may the level of work necessary to assess the sufficiency and appropriateness of evidence to address the objectives. The concepts of audit risk and significance assist auditors in evaluating the audit evidence. Professional judgment assists auditors in determining the sufficiency and appropriateness of evidence taken as a whole. Interpreting, summarizing, or analyzing evidence is typically used in determining the sufficiency and appropriateness of evidence and in reporting the results of the audit work.
8.96
When auditors use information that audited entity officials provided as part of their evidence, auditors may find it necessary to test management’s procedures to obtain assurance, perform direct testing of the information, or obtain additional corroborating evidence. The nature, timing, and extent of the auditors’ procedures will depend on the significance of the information to the audit objectives and the nature of the information being used. Using a risk-based approach, auditors may consider additional procedures if they become aware of evidence that conflicts with that provided by management. In their overall assessment, auditors may document how they resolved situations involving conflicting evidence.76
8.97
Auditors may request that management provide written representations as to the accuracy and completeness of information provided.
8.98
The nature, timing, and extent of audit procedures to assess sufficiency and appropriateness are affected by the effectiveness of the audited entity’s internal controls over the information, including information systems controls, and the significance of the information and the level of detail presented in the auditors’ findings and conclusions in the context of the audit objectives. The sufficiency and appropriateness of computer-processed information is assessed regardless of whether this information is provided to auditors or auditors independently extract it. Assessing the sufficiency and appropriateness of computer-processed information includes considering the completeness and accuracy of the data for the intended purposes.
Sufficiency
8.99
Sufficiency is a measure of the quantity of evidence used to support the findings and conclusions related to the audit objectives.
8.100
When appropriate, auditors may use statistical methods to analyze and interpret evidence to assess its sufficiency.
8.101
The sufficiency of evidence required to support the auditors’ findings and conclusions is a matter of the auditors’ professional judgment. The following presumptions are useful in judging the sufficiency of evidence.
The greater the audit risk, the greater the quantity and quality of evidence required.
Stronger evidence may allow less evidence to be used.
Appropriateness
8.102
Appropriateness is the measure of the quality of evidence that encompasses the relevance, validity, and reliability of evidence used for addressing the audit objectives and supporting findings and conclusions.
Relevance refers to the extent to which evidence has a logical relationship with, and importance to, the issue being addressed.
Validity refers to the extent to which evidence is a meaningful or reasonable basis for measuring what is being evaluated. In other words, validity refers to the extent to which evidence represents what it is purported to represent.
Reliability refers to the consistency of results when information is measured or tested and includes the concepts of being verifiable or supported. For example, in establishing the appropriateness of evidence, auditors may test its reliability by obtaining supporting evidence, using statistical testing, or obtaining corroborating evidence.
Having a large volume of evidence does not compensate for a lack of relevance, validity, or reliability.
8.103
The degree of assurance associated with a performance audit is strongly associated with the appropriateness of evidence in relation to the audit objectives. Examples follow.
The audit objectives might focus on verifying specific quantitative results presented by the audited entity. In these situations, the audit procedures would likely focus on obtaining evidence about the accuracy of the specific amounts in question. This work may include the use of statistical sampling.
The audit objectives might focus on the performance of a specific program or activity in the audited entity. In these situations, the auditors may be provided information that the audited entity compiled in order to satisfy the audit objectives. The auditors may find it necessary to test the quality of the information, which includes both its validity and reliability.
The audit objectives might focus on information that is used for widely accepted purposes and obtained from sources generally recognized as appropriate. For example, economic statistics issued by government agencies for purposes such as adjusting for inflation, or other such information issued by authoritative organizations, may be the best information available. In such cases, it may not be practical or necessary for auditors to perform procedures to verify the information. These decisions call for use of professional judgment based on the nature of the information, its common usage or acceptance, and how it is being used in the audit.
The audit objectives might focus on comparisons or benchmarking between various government functions or agencies. These types of audits are especially useful for analyzing the outcomes of various public policy decisions. In these cases, auditors may perform analyses, such as comparative statistics of different jurisdictions or changes in performance over time, where it would be impractical to verify the detailed data underlying the statistics. Clear disclosure of the extent to which comparative information or statistics were evaluated or corroborated will likely be necessary to place the evidence in context for report users.
The audit objectives might focus on trend information based on data that the audited entity provided. In this situation, auditors may assess the evidence by using overall analytical tests of underlying data, combined with knowledge and understanding of the systems or processes used for compiling information.
The audit objectives might focus on identifying emerging and crosscutting issues using information that audited entities compiled or self-reported. In such cases, it may be helpful for the auditors to consider the overall appropriateness of the compiled information along with other information available about the program. Other sources of information, such as inspector general reports or other external audits, may provide the auditors with information regarding whether any unverified or self-reported information is consistent with or can be corroborated by these other external sources of information.
8.104
In terms of its form and how it is collected, evidence may be categorized as physical, documentary, or testimonial. Physical evidence is obtained by auditors’ direct inspection or observation of people, property, or events. Such evidence may be documented in summary memos, photographs, videos, drawings, charts, maps, or physical samples. Documentary evidence is already existing information, such as letters, contracts, accounting records, invoices, spreadsheets, database extracts, electronically stored information, and management information on performance. Testimonial evidence is obtained through inquiries, interviews, focus groups, public forums, or questionnaires. Auditors frequently use analytical processes, including computations, comparisons, separation of information into components, and rational arguments, to analyze any evidence gathered to determine whether it is sufficient and appropriate. Evidence may be obtained by observation, inquiry, or inspection. Each type of evidence has its own strengths and weaknesses. The following contrasts are useful in judging the appropriateness of evidence. However, these contrasts are not adequate in themselves to determine appropriateness. The nature and types of evidence used to support auditors’ findings and conclusions are matters of the auditors’ professional judgment based on the audit objectives and audit risk.
Evidence obtained when internal control is effective is generally more reliable than evidence obtained when internal control is weak or nonexistent.77
Evidence obtained through the auditors’ direct physical examination, observation, computation, and inspection is generally more reliable than evidence obtained indirectly.
Examination of original documents is generally more reliable than examination of copies.
Testimonial evidence obtained under conditions in which persons may speak freely is generally more reliable than evidence obtained under circumstances in which the persons may be intimidated.
Testimonial evidence obtained from an individual who is not biased and has direct knowledge about the area is generally more reliable than testimonial evidence obtained from an individual who is biased or has indirect or partial knowledge about the area.
Evidence obtained from a knowledgeable, credible, and unbiased third party is generally more reliable than evidence obtained from management of the audited entity or others who have a direct interest in the audited entity.
8.105
Testimonial evidence may be useful in interpreting or corroborating documentary or physical information. Documentary evidence may be used to help verify, support, or challenge testimonial evidence.
8.106
Surveys generally provide self-reported information about existing conditions or programs. Evaluating the survey design and administration assists auditors in evaluating the objectivity, credibility, and reliability of the self-reported information.
8.107
When sampling is used, the appropriate selection method will depend on the audit objectives. When a representative sample is needed, the use of statistical sampling approaches generally results in stronger evidence than that obtained from nonstatistical techniques. When a representative sample is not needed, a targeted selection may be effective if the auditors have isolated risk factors or other criteria to target the selection.
Overall Assessment of Evidence
Requirements: Overall Assessment of Evidence
8.108
Auditors should perform and document an overall assessment of the collective evidence used to support findings and conclusions, including the results of any specific assessments performed to conclude on the validity and reliability of specific evidence.
8.109
When assessing the overall sufficiency and appropriateness of evidence, auditors should evaluate the expected significance of evidence to the audit objectives, findings, and conclusions; available corroborating evidence; and the level of audit risk. If auditors conclude that evidence is not sufficient or appropriate, they should not use such evidence as support for findings and conclusions.
8.110
When the auditors identify limitations or uncertainties in evidence that is significant to the audit findings and conclusions, they should perform additional procedures, as appropriate.
Application Guidance: Overall Assessment of Evidence
8.111
Professional judgments about the sufficiency and appropriateness of evidence are closely interrelated, as auditors interpret the results of audit testing and evaluate whether the nature and extent of the evidence obtained is sufficient and appropriate.
8.112
Sufficiency and appropriateness of evidence are relative concepts, which may be thought of as a continuum rather than as absolutes. Sufficiency and appropriateness are evaluated in the context of the related findings and conclusions. For example, even though the auditors may identify some limitations or uncertainties about the sufficiency or appropriateness of some of the evidence, they may nonetheless determine that in total there is sufficient, appropriate evidence to support the findings and conclusions.
8.113
The steps to assess evidence may depend on the nature of the evidence, how the evidence is used in the audit or report, and the audit objectives.
Evidence is sufficient and appropriate when it provides a reasonable basis for supporting the findings or conclusions within the context of the audit objectives.
Evidence is not sufficient or appropriate when (1) using the evidence carries an unacceptably high risk that it could lead auditors to reach an incorrect or improper conclusion; (2) the evidence has significant limitations, given the audit objectives and intended use of the evidence; or (3) the evidence does not provide an adequate basis for addressing the audit objectives or supporting the findings and conclusions.
8.114
Evidence has limitations or uncertainties when its validity or reliability has not been assessed or cannot be assessed, given the audit objectives and the intended use of the evidence. Limitations also include errors identified by the auditors in their testing.
8.115
Additional procedures that could address limitations or uncertainties in evidence that are significant to the audit findings and conclusions include
seeking independent, corroborating evidence from other sources;
redefining the audit objectives or the audit scope to eliminate the need to use the evidence;
presenting the findings and conclusions so that the supporting evidence is sufficient and appropriate and describing in the report the limitations or uncertainties with the validity or reliability of the evidence, if such disclosure is necessary to avoid misleading the report users about the findings or conclusions; and
determining whether to report the limitations or uncertainties as a finding, including any related significant internal control deficiencies.
Findings
Requirements: Findings
8.116
As part of a performance audit, when auditors identify findings, they should plan and perform procedures to develop the criteria, condition, cause, and effect of the findings to the extent that these elements are relevant and necessary to achieve the audit objectives.
8.117
Auditors should consider internal control deficiencies in their evaluation of identified findings when developing the cause element of the identified findings when internal control is significant to the audit objectives.
Application Guidance: Findings
8.118
Findings may involve deficiencies in internal control; noncompliance with provisions of laws, regulations, contracts, and grant agreements; or instances of fraud.
8.119
Given the concept of accountability for use of public resources and government authority, evaluating internal control in a government environment may also include considering internal control deficiencies that result in waste or abuse. Because the determination of waste and abuse is subjective, auditors are not required to perform specific procedures to detect waste or abuse in performance audits. However, auditors may consider whether and how to communicate such matters if they become aware of them. Auditors may also discover that waste or abuse are indicative of fraud or noncompliance with provisions of laws, regulations, contracts, and grant agreements.
8.120
Waste is the act of using or expending resources carelessly, extravagantly, or to no purpose. Importantly, waste can include activities that do not include abuse and does not necessarily involve a violation of law. Rather, waste relates primarily to mismanagement, inappropriate actions, and inadequate oversight.
8.121
The following are examples of waste, depending on the facts and circumstances:
Making travel choices that are contrary to existing travel policies or are unnecessarily extravagant or expensive.
Making procurement or vendor selections that are contrary to existing policies or are unnecessarily extravagant or expensive.
8.122
Abuse is behavior that is deficient or improper when compared with behavior that a prudent person would consider reasonable and necessary business practice given the facts and circumstances, but excludes fraud and noncompliance with provisions of laws, regulations, contracts, and grant agreements. Abuse also includes misuse of authority or position for personal financial interests or those of an immediate or close family member or business associate.
8.123
The following are examples of abuse, depending on the facts and circumstances:
Creating unneeded overtime.
Requesting staff to perform personal errands or work tasks for a supervisor or manager.
Misusing the official’s position for personal gain (including actions that could be perceived by an objective third party with knowledge of the relevant information as improperly benefiting an official’s personal financial interests or those of an immediate or close family member; a general partner; an organization for which the official serves as an officer, director, trustee, or employee; or an organization with which the official is negotiating concerning future employment).
8.124
Criteria: To develop findings, criteria may include the laws, regulations, contracts, grant agreements, standards, measures, expected performance, defined business practices, and benchmarks against which performance is compared or evaluated. Criteria identify the required or desired state or expectation with respect to the program or operation. The term program includes processes, projects, studies, policies, operations, activities, entities, and functions. Criteria provide a context for evaluating evidence and understanding the findings, conclusions, and recommendations in the report.
8.125
Condition: Condition is a situation that exists. The condition is determined and documented during the audit.
8.126
Cause: The cause is the factor or factors responsible for the difference between the condition and the criteria, and may also serve as a basis for recommendations for corrective actions. Common factors include poorly designed policies, procedures, or criteria; inconsistent, incomplete, or incorrect implementation; or factors beyond the control of program management. Auditors may assess whether the evidence provides a reasonable and convincing argument for why the stated cause is the key factor contributing to the difference between the condition and the criteria.
8.127
Effect or potential effect: The effect or potential effect is the outcome or consequence resulting from the difference between the condition and the criteria. When the audit objectives include identifying the actual or potential consequences of a condition that varies (either positively or negatively) from the criteria identified in the audit, effect is a measure of those consequences. Effect or potential effect may be used to demonstrate the need for corrective action in response to identified problems or relevant risks.
8.128
The elements needed for a finding are related to the objectives of the audit. Thus, a finding or set of findings is complete to the extent that the audit objectives are addressed and the report clearly relates those objectives to the elements of a finding. For example, an audit objective may be to determine the current status or condition of program operations or progress in implementing legislative requirements, and not the related cause or effect. In this situation, developing the condition would address the audit objective, and developing the other elements of a finding would not be necessary.
8.129
The cause of a finding may relate to an underlying internal control deficiency. For example, auditors conducting a compliance audit may find that an audited entity has not complied with certain legislation. Upon further evaluation, the auditors may find the root cause of the finding to be that one of the entity’s control activities was not properly designed. In this case, the finding would be an instance of noncompliance, but the cause of the finding would be an internal control deficiency.
8.130
Considering internal control in the context of a comprehensive internal control framework, such as Standards for Internal Control in the Federal Government or Internal Control—Integrated Framework,78 can help auditors to determine whether underlying internal control deficiencies exist as the root cause of findings. When the audit objectives include explaining why a particular type of positive or negative program performance, output, or outcome identified in the audit occurred, the underlying deficiencies are referred to as cause. Identifying the cause of problems may assist auditors in making constructive recommendations for correction. Auditors may identify deficiencies in program design or structure as the cause of deficient performance. Auditors may also identify deficiencies in internal control that are significant to the subject matter of the performance audit as the cause of deficient performance. In developing these types of findings, the deficiencies in program design or internal control would be described as the cause. Often the causes of deficient program performance are complex and involve multiple factors, including fundamental, systemic root causes.
8.131
When the audit objectives include estimating the extent to which a program has caused changes in physical, social, or economic conditions, “effect” is a measure of the program’s impact. In this case, effect is the extent to which positive or negative changes in actual physical, social, or economic conditions can be identified and attributed to the program.
See para. 8.105 for a discussion of the relationship between testimonial and documentary evidence.↩︎
See paras. 8.39 through 8.67 for a discussion of internal control.↩︎
The COSO Framework and the Green Book provide suitable and available criteria against which management may evaluate and report on the effectiveness of the entity’s internal control. The Green Book may be adopted by entities beyond those federal entities for which it is legally required, such as state, local, and quasi-governmental entities, as well as other federal entities and not-for-profit organizations, as a framework for an internal control system.↩︎