Processing simulator data...
If this banner does not disappear after 10 seconds, your browser settings may be incompatible with the simulator.
Please ask your IT department to whitelist this address, or try accessing the site via a non-managed device.

ID Verification Controls Simulator

An Interactive Illustration of Concepts identified by the June 2021 JFMIP Expert Panel on Identity Verification to Prevent Improper Payments

Simulation Settings

Who is applying to the hypothetical program?

How will the hypothetical program operate?

Note: selecting a uniform control tier structure may limit your system components available for selection below.
How will the hypothetical program design each tier of controls?

Would you like to compare outcomes of different system configurations?

Note:
  • A full system comparison allows users to see the differences in all transaction outcomes between the original and reconfigured settings. A mid-program reconfiguration applies different configurations based on the time of a transaction, and shows the differences in outcomes for transactions occuring in the second half of the hypothetical program's duration.
  • For the purpose of this simulation, choosing a reconfiguration does not directly affect administrative costs, applicant burden, nor program reputation, though this may not be the case in real program scenarios.

Would you like to re-randomize the simulation?

Note: Each random state is tied to a numeric value, also known as a seed. Changing the seed to a new number in the box below will cause all randomized elements in the simulation to be re-randomized.

Introduction

Click here to show/hide the video transcript

Federal benefits programs can help people in need by issuing payments to eligible applicants to make sure that the right payments are going to the right people. Programs can use identity verification controls to make sure that applicants are, in fact, who they say they are. In their applications. So the Joint Financial Management Improvement Program began an initiative focused on identity verification controls as a way to mitigate the risk of improper payments and to increase federal program payment integrity.

For this phase of the initiative, we convened an expert panel to discuss the concepts and ideas around identity verification controls. Our report presents key practices based on those ideas and potential actions that agencies could take to better verify applicants identities. The report also includes considerations for developing a framework that may facilitate the adoption and implementation of identity verification controls.

Throughout federal programs. Along with the report, we published an interactive tool that illustrates some of its concepts using synthetic data, neither based on real applicant nor organization data. The hypothetical program scenarios in the tool will allow you to strengthen your understanding of how identity verification controls function. To think about the tradeoffs that you may face when implementing different sets of identity verification controls, to gather in a system and to consider how identity verification controls may be relevant within your organizational context.

The simulator is just an illustration, but you can leverage the insights you gain from using it. To start conversations about how identity verification controls might be appropriate for your organization or program. Check out the next video to learn how to use the simulator.

The Identity (ID) Verification Controls Simulator is an interactive illustration of concepts identified throughout the corresponding Joint Financial Management and Improvement Program (JFMIP) Report, which program offices or a central management entity could consider for the purposes of enhancing identity verification, and potentially reducing improper payments to increase payment integrity. In this interactive illustration, you are the manager of a hypothetical federal program that makes one-time payments to applicants. These applicants may either be proper (someone who accurately provides their own identity) or improper applicants (someone who provides a misrepresented identity) in their attempts to receive the one-time payments.

This illustration demonstrates the importance of considering concepts from the JFMIP report both individually and in tandem when developing and implementing internal controls to address the risk of improper payments. It demonstrates how program context (who is applying to the hypothetical program) and internal controls responses (how the hypothetical program will operate) may impact the hypothetical program’s ability to successfully identify and serve proper applicants, while successfully identifying improper applicants and preventing improper payments to them. Your choices will result in three categories of hypothetical outcomes:

  • The proportion of both improper–and proper–payments prevented
  • The program’s administrative costs
  • The program’s reputation, including…
    • The burden placed on proper applicants
    • The number of proper applicants deterred from completing the application process
    • The security and privacy of applicant information

You can use the simulator to explore simplified examples of ways that a hypothetical program’s context and its control responses may impact these outcomes. However, as we indicate by emphasizing that the simulation is highly stylized for the purposes of creating an interpretable, hypothetical illustration, it should not be used to support management and internal control decisions of real world programs. The italicized text is directly from the report, where these concepts are explained in greater detail.

Overview of Hypothetical Choices and Impacts within the Simulation

The “Simulation Settings” section offers several different options for users configure the simulation, along with a summary of the impacts those choices have on the hypothetical program outcomes. The navigation tab for the main panel allows users to view more information about the configuration options. For more information on the simulator interface, please use the tab labeled, “Understanding the Simulator Interface.”

These options allow you to make choices to change the hypothetical program context and internal control response, and to observe illustrative changes in relevant outcomes. The configuration options reflect the key practices the JFMIP identified throughout the report based on the expert panel discussion including:

  • Forming data-sharing partnerships among federal or state agencies, such as Do Not Pay
  • Applying for additional program resources, such as the Technology Modernization Fund
  • Introducing risk-based transaction management for internal controls in response to federal-benefit applications
  • Taking steps to secure and encrypt applicant data

The simulator similarly describes outcomes raised during the expert panel, as described above.

See the tab, “Tutorial Examples,” for walkthroughs on some of the available choices and impacts within the simulation.

Simulation Methodology and Data

To build this simulation, we built upon the methodology described in the JFMIP report (https://www.cfo.gov/jfmip/payment-integrity-initiative). Specifically, we used what we learned from the report to develop initial relationships and assumptions about each of the options in the simulation. We shared this initial work with participants from the panel of experts discussed in the report to gather more information and incorporate feedback to ensure these assumptions were minimally reasonable and plausible. We also conducted several user tests of the simulation with Government Accountability Office (GAO) staff, JFMIP experts, and cold readers from JFMIP agencies to assess the clarity, usefulness, and presentation of the simulation, and made changes to improve these areas, as appropriate.

This simplified representation of a purely fictitious program and its applicants situates practices from the ID Verification Report in a hypothetical scenario, which is not based on any real program context nor implementation of a specific internal controls system. As such, while it represents a reasonable illustration of important concepts and tradeoffs, the simulation is not intended to be representative of federal programs, and should not be used for real world decisions about specific programs.

Each time the simulation is run, a synthetic dataset of 1,000 transactions between programs and applicants is generated based on the options you select. The underlying data in a given run of the simulation may be accessed under the tab labeled “Can you improve the hypothetical program’s performance?” on the “Data Viewer” sub-tab. The Data Viewer allows you to generate an auditable narrative of how each transaction was generated, which represents a granular, distinct, qualitative experience for each applicant in the program. The full technical details of the simulation are discussed in the “Technical White Paper” tab.

Understanding the Simulator Interface

Click here to show/hide video caption

Welcome to the Identity Verification Control Simulator.

This video will show you how to use it in conjunction with the report to think about the tradeoffs that you may face when implementing different sets of identity verification controls together in a system.

When you first launch the simulator you’ll see three main sections, the simulation settings side panel, the navigation bar, and the main content panel, which shows the information selected in the navigation bar. The side panel is where you can change the inputs related to who is applying to the hypothetical program and how the hypothetical program operates, in addition to quickly viewing a summary of the outcomes. Advanced users can also compare multiple configurations or re-randomize the simulated data. The navigation bar has four main tabs background, Can you improve the hypothetical program’s performance? Who is applying to the hypothetical program and how will the hypothetical program operate?

Each tab has two or more subtabs underneath it. There’s a map to the tab structure and the background subtab, understanding the simulator interface. The background subtab, tutorial examples walks you through some examples to better explain the tools mechanics.

Results and data appear under the tab labeled: Can you improve the hypothetical program’s performance? The outcomes of your side panel choices are summarized across the top of this section in four key metrics.

The proportion of improper payments stopped. The proportion of proper payments stopped. The administrative costs and the participant satisfaction. These metrics can help you understand the effects of experimenting with different program contexts and control choices. For example, if you select the most stringent controls, you may stop more improper payments relative to if you had selected less stringent control choices. The accuracy of these controls may mean that you’re also able to issue more proper payments. However, the rigor of these controls may cause some would be proper applicants to give up on their applications Along with increasing administrative costs, the other two main tabs provide information for configuration options in the side panel. These tabs contain relevant context from the report as well as information on how we built the concepts into the simulator.

There are three core components to the simulation interface: the Navigation Bar, the Simulation Settings, and the content itself.

Using the Navigation Bar

There are four main tabs in the simulator, each with their own sub-tabs. The content available on these pages is informational, and will provide info that directly relates to the sections in the Simulation Settings that share the same header. For example, the information in the tab labeled, “Who is applying to the hypothetical program?,” corresponds to the box in Simulation Settings that shares the same label. The full navigable contents of the simulation are as follows:

  • Background
    • Getting Started
    • Understanding the Simulator Interface
    • Metrics
    • Tutorial Examples
    • Technical White Paper
  • Can you improve the hypothetical program’s performance?
    • Results
    • Data Viewer
  • Who is applying to the hypothetical program?
    • Socioeconomic Vulnerability
    • Misrepresentation Rate and Type
  • How will the hypothetical program operate?
    • System Walkthrough
    • Control Tiers: Risk-based vs. Uniform
    • A) Digital Footprint Monitoring
    • B) Data Sharing Agreement
    • C) Predictive Risk Model
    • D) Design of Control Tiers
    • Budget
    • Security

Using the Simulation Settings

The first box under Simulation Settings, “Can you improve the hypothetical program’s performance?,” is a summary of the results, which are expanded in the corresponding informational tab. The remaining boxes all contain options that affect the simulation results. Any of the settings in “Who is applying to the hypothetical program?” and “How will the hypothetical program operate” have corresponding informational tabs.

Neither of the final two boxes: “Would you like to compare outcomes of different system configurations?” and “Would you like to re-randomize the simulation?” correspond with informational tabs, since they are specifically for the purposes of the simulation. The reconfiguration options allow users to choose a second set of controls, which can be used to test how the same applicants would respond to different controls. The controls can be applied at different points of time, to show what the affect would be of a change in controls midway through the duration of the hypothetical program. The re-randomize option allows users to generate a full new set of transactions during the same time period. This will allow users to compare results of the same selected controls against a different set of synthetically generated transactions.

Overview of Metrics

The following italicized text is from the report.

According to The National Institute of Standards and Technology (NIST), organizations that implement identity proofing generally seek to balance cost, convenience, and security for both the provider and the individual, which may include assessing the trade-off between false positives and false negatives. NIST also explains that, taking steps to reduce false negatives in identity verification can result in increased risk of false positives and user abandonment.

Several panelists noted that implementing identity-verification controls can result in false positives and false negatives, which could create a burden on both the program office and the applicants. In the context of detecting misrepresented identities, a “false positive” is incorrectly determining that a transaction is associated with a misrepresented identity, when in reality, an applicant has not misrepresented identity. For example, a control might fail to recognize a match between an applicant’s current appearance and an older driver’s license photo on file and incorrectly determine that the applicant is not who the applicant claims to be. A “false negative” is incorrectly determining that a transaction is not associated with a misrepresented identity, when in reality, the applicant has misrepresented identity. For example, identity thieves might alter their appearances digitally with a video filter so that they resemble their victims; controls might fail to detect such alteration.

The costs of false negatives and false positives will vary by program and context. If possible, program offices can explicitly determine the cost of false positives and false negatives and make this trade-off for their programs. Program offices can decide on a threshold based on their tolerance for false negatives and their willingness to commit resources, such as the staffing resources needed to sift through a large amount of false positive results to prevent an improper payment.

For more information, please visit page 50 in the report.

Metrics in the Simulation

Administrative Costs

All controls in the simulation introduce an administrative cost-per-applicant to the program. More rigorous, higher-tier controls are generally costlier to administer. Therefore, total administrative costs for the program are a function of applicant volume as well as the rate at which the control system places applicants into higher control tiers.

True Positive and False Positive Rates

A “Positive” determination occurs when the simulator determines an applicant to be improper (i.e. misrepresenting their identity). A “True positive” means the simulator has done this correctly, while a “false positive” means the system has incorrectly identified a proper applicant as being improper. An identity verification control system will inevitably subject some proper applicants to more-rigorous controls. This can be the result of an error in an initial predictive risk model or as a result of an applicant’s failing a control check (e.g., not correctly remembering a past address). The simulation subjects such applicants to further and more-rigorous controls, such as biometric and/or in-person verification, than those to which they should have been subjected. “True Positive Rate” in the simulation is the proportion of positive determinations that the system correctly identifies. Similarly, the false positive rate is the proportion that the system incorrectly identifies as improper. A system with more and better controls will generally have a greater ability to detect misrepresented identities correctly. Consequently, the system may have both a greater administrative cost and burden on applicants. A program with a less-effective model for identifying applicant risk may end up with a high false-positive rate. If this hypothetical program also has a high rate of socioeconomically vulnerable applicants, the result may be deterring an excessive number of proper applicants from completing applications for benefits.

True Negative and False Negative Rates

A “Negative” determination occurs when the simulator determines an applicant to be proper (i.e. not misrepresenting their identity). A “True Negative” means the simulator correctly identified that the applicant is not misrepresenting their identity. A “False Negative” means the simulator mis-identified an applicant as proper when they were misrepresenting their identity. “True negative rate” is the proportion of True Negatives the system correctly identifies, while “False Negative Rate” is the proportion of false negatives that the system incorrectly identifies.
Identity vs. Controls Response

Applicant Burden

In the simulation, applicants exposed to more rigorous or time consuming control checks are assumed to become more frustrated with the program. In the case of proper transactions, this harms the program’s reputation and may eventually cause applicants to exit the process out of frustration. In the case of improper transactions, applicant burden may also cause bad actors to exit the program; however, the applicant burden for improper transactions does not affect the program’s reputation.

Tutorial Examples

The following tutorial examples are intended demonstrate how the simulation works, and not to suggest any preference in which options are selected. As a reminder, each program will have a different context and implementation of the various concepts illustrated in the simulation, and individualized assessments specific to each program are necessary to support informed decisions on whether and how to implement specific controls. For example, the simulation illustrates an implementation of digital footprint monitoring that attempts to eliminate bot-driven applications. However, this type of monitoring could also be used, for example, as a tool to assess the risk of applications from bots without eliminating them directly. The simulation relies on the stylization of this control in order to represent one possible implementation amongst many that could reasonably make sense for different real-world programs.


Uniform vs. Risk-based Controls

By default, the system configuration starts with a uniform, low control tier system selected. Comparing the uniform system with a risk-based system, you will see that any of the system components that help to identify risk are hidden. When using uniform control system, the same controls are applied to every application, regardless of the risk; therefore, the simulator does not give you the option to add these system components to a uniform configuration, where they would increase the cost, but offer no change in outcome.

Without changing any other parameters, let us look at the summary level results shown in the side panel and at the results shown on the results page.

As you can see, we have prevented a proportion of improper payments. Depending on your own sensitivity, this may be acceptable or unacceptable – it will vary based on the circumstances of each hypothetical program.

Yet, as a consequence of applying only the low tier controls — which in the simulation are less accurate than the high tier controls — we have also prevented a relatively high proportion of legitimate applicants from receiving payment. Again, depending on your risk sensitivity, this may be acceptable, or this may be a critical program failure.

Why might these proper applicants be prevented from receiving payment?

  • There is a higher false positive rate at a single, low tier of controls, with no second opportunity to prove legitimacy.
  • Some proper applicants are deterred, but not many, since these controls are less invasive.

We select different options in the simulator to try improve the hypothetical program's outcomes. let us select the re-configure option and compare the original low tier with the high tier. You will see that the simulator updates these metrics based on the final outcome of the selected reconfiguration. In order to do a full system comparison, let us set the re-configuration date to the first day of the program, so that we can see what would happen if an entirely different program configuration was applied to the same population.

Relative to the low tier system, we prevented a larger proportion of improper payments. Higher tier controls are better at detecting improper attempts — and also deterring them — since bad actors would know that they were more likely to get caught with these attempts.

Also note: we decreased the proportion of legitimate applicants who we otherwise would prevent from receiving payment using the lower tier of controls. While we might more readily deter applicants at this level, the increased accuracy of the controls helps to reduce the false positives, so overall, we have increased the proportion of legitimate payments made.

What would happen if we used the risk-based system instead? let us set our original configuration to the "high" tier, and the re-configuration to the risk-based option.

You should see that the risk based option prevents marginally fewer improper payments, while further reducing the number of legitimate payments which are deterred. Using the risk-based option, we can triage individuals to different control tiers, which ideally, allows us to make payments more easily to lower risk legitimate applicants, while delegating those that are higher risk applications to higher tiers.


Security Impacts on Satisfaction

Now that you have seen the difference between uniform and risk-based systems, let us take into account another metric used in the simulator: proper applicants' satisfaction.

The simulation assumes that applicants in the simulation who are not paid will never be satisfied with the program. Those who get paid in the simulation are more likely to be satisfied, but can still be dissatisfied, based on their experience in the program. let us consider how this mechanic functions in a uniform, low control tier system.

  • Pros: Applicants who are paid at the lower tier are more likely to be satisfied than individuals paid at higher tiers, due to the lower burden.
  • Cons: Conversely, we are more likely to prevent applicants from receiving payment at lower control tiers, due to the greater false positive rate, even though they are less likely to be deterred by the burden of controls.

Similarly, we can make all applicants less satisfied if their data was potentially compromised. One concern discussed in the expert panel was that collecting and centralizing data via this system would increase a program's risk of hacking attempts.

In the simulation, we have designed security risks such that each transaction is associated with a very small risk of compromising the data included in the system. Imagine that an applicant receives an email prompting them to apply or finish applying to the program, but that email is from a bad actor. Through the use of malware, the bad actor may be able to access the system as if they were the actual user, and to compromise the database from past an initial initial layer of security. In other words, as we have chosen to represent it in the simluation, if a security breach occurs, all data through the date of the security breach is considered "compromised," which dissatisfies the applicants, since they have the additional concern of resolving any issues resulting from the possibly compromised data.

Starting with the baseline of these results:

then enabling program reconfiguration from day 1, and increasing the security quality beyond the basic level, we will see an increase in overall applicant program satisfaction.

Yet, if we were to reconfigure the program to a later date, leaving us with lower security up until that point, a hacking incident may have occurred, which would decrease overall applicant satisfaction.

Outcome of ID Verification Control Choices


The plot below represents a population of 1,000 hypothetical applicants (generated using synthetic data) who are seeking payments from the hypothetical benefits program. Each applicant requests a payment of between $100 and $10,000 on a log distribution (i.e., most are closer to $2,000). Each time a user runs the simulation, the tool randomly generates attributes such as socioeconomic vulnerability and identity misrepresentation according to the parameters selected by the users. Each line represents the running total of dollars per type of transaction. Please note that the y-axis scale changes to include the full range of values in the display window.

The table below shows the 1,000-person hypothetical applicant population, based on the result of their participation in the program in terms of A) dollar value of payments made to applicants, B) dollar value of administrative costs associated with internal controls for these payments, and C) the number of individuals in each applicant group.

The table below shows the 1,000 -person hypothetical applicant population, based upon the result of their participation in the program in terms of A) the number of escalations each applicant experienced in processing, B) dollar value of administrative costs associated with internal controls for these payments, and C) the number of individuals in each applicant group.

The plot below shows the proportion of payments made to proper and improper applicants based on the final control tier that the system applied to each transaction. If Digital Footprinting is enabled, a 'Tier 0' will appear as a final tier, and represents any application that is flagged by the Digital Footprint. Within the hypothetical scenario, these applications are rejected before any other internal controls are applied.

The table below represents the socioeconomic vulnerability of proper applicants whose transactions were either stopped or prevented by the user-selected controls.

The table below represents the program satisfaction of proper applicants, where each applicant can contribute up to 1 / 1,000th of a point to a group-wide scale, ranging from -1 to +1. On the satisfaction scale, -1 represents a population that is very dissatisfied with the experience in the program; +1 represents a population that is very satisfied. Applicants can receive payments and be unsatisfied, depending on their individual applicant burden. In the event of a security incident, the satisfaction level of all affected applicants decreases by an amount corresponding to the level of data that their assigned control tier required them to provide to the hypothetical program.

Note: Generating narrative logs can take a moment to load, depending on the simulation settings.

Overview of Socioeconomic Vulnerability

The following italicized text is from the report.

Certain segments of the population, such as socioeconomically vulnerable individuals, may face an increased burden when verifying their identities. This may occur because these individuals may not have the required data elements or documentation or may not have the means to supply the required information.

Consider the following examples described in the report:

  • Those who are homeless may lack driver’s licenses or physical ID because their states’ motor vehicle agencies require a physical address in order to issue a driver’s license. In this case, a verification process that relies solely on individuals’ ability to provide evidence of something they have could be problematic.
  • Socioeconomically vulnerable individuals may lack transportation, or the ability to take time off from their jobs, to complete steps necessary for in-person verification.
  • Additionally, they may lack the technology, such as a smartphone or access to the internet, to complete remote verification. In this case, a verification process that relies only on individuals’ ability to provide evidence of something they are could be problematic.

For more information, please visit page 49 in the report.

Socioeconomic Vulnerability in the Simulation

Simulation users can select whether the program has a higher or lower proportion of applicants who face socioeconomic vulnerabilities.

For the purposes of this simulation, socioeconomic vulnerability refers to one’s ability to adequately respond to and cope with changes to their social or economic well-being. Control processes that place burdens on applicants may more negatively affect those with a higher socioeconomic vulnerability. The additional challenges these applicants face, may cause them to either 1) choose not to participate in the program, or 2) deter them from responding to controls the program poses. In either case, the hypothetical benefits program would not make payments to these potential recipients, which could harm the program’s ability to effectively achieve its goals.

Mechanics in Our Model

The mechanics outlined below are a simplification for our model and only for illustrative purposes. While we have taken steps to ensure this illustration is reasonably plausible, they do not comprehensively represent the full range of real-world outcomes. As such, this model should not be used as a basis for decisions about real world programs.

  • Within the simulation, proper applicants with higher vulnerability have a higher probability of being deterred by the application process, as explained in the technical whitepaper.
  • Moreover, in multi-tiered control systems, individuals who fail ID verification controls are escalated to the next control tier, where they face an increased probability of deterrence than if they were originally assigned that tier. This illustrates a potential compounding effect of applicant burdens which may disproportionally impact proper applicants who have the highest socioeconomic vulnerability.
  • Click here to show/hide technical details
    • By default, the model assumes roughly 16% of applicants will be very vulnerable. Selecting "High" average vulnerability will increase this to about 50%. Selecting "low" will decrease this to about 1%.
    • We represent socioeconomic vulnerability on an index of between approximately 0 and 6, representing the least and most socioeconomically vulnerable populations, respectively.
    • The user may select the average level of socioeconomic vulnerability in the program. By default, vulnerability is 3 (standard deviation = 1 ). Selecting "low" reduces the mean to 2; selecting "high" increases the mean to 4.
    • A proper applicant will be deterred at a rate equal to 5% * applicant burden * socioeconomic vulnerability, where applicant burden is determined by the control tier into which the applicant is placed.
    • Applicants who are "escalated" a control tier because they fail ID verification controls at their current level must bear the applicant burden of additional controls at the raised control tier, and thus face an additional chance of being deterred. In addition to the normal applicant burden at their escalated control tier, they face a 5% increase for each such escalation due to compounding frustration. Thus, an applicant escalated two times to a tier where the burden is originally 5 would instead experience a burden of 1 + (0.05 * 2 ) * 5 = 1.5.
(Click to read more in the technical whitepaper)

Overview of Misrepresentation Rate and Type

The following italicized text is from the report.

Understanding its susceptibility to different types of misrepresentation allows a program office to select the most appropriate verification controls to mitigate those risks. Identity thieves can obtain sensitive personal information through various methods, such as using phishing to trick individuals or employees of an organization into sharing their own or others’ sensitive personal information. Identity theft also can occur as a result of the loss or theft of data (a lost or stolen wallet or a thief digging through household trash). Because of the varying nature of how identity theft occurs, identity thieves may have greater or lesser amounts of information available.

Improper payments, fraud, and fraud risk are related, but distinct, concepts. While unintentional error may cause improper payments, fraud involves obtaining something of value through willful misrepresentation. Whether an act is fraudulent is determined through the judicial or other adjudicative system. Fraud risk exists when individuals have an opportunity to engage in fraudulent activities, have an incentive or are under pressure to commit fraud, or are able to rationalize committing fraud.

For more information, please visit page 25 in the report.

Misrepresentation Rate in the Simulation

As described above, the percentage and nature of transactions that intentionally misrepresent identities may vary significantly by program and over time within the same program. As such, the simulation allows users to select the percent of program transactions that intentionally misrepresent an identity. Further, simulation users can adjust the percent of misrepresentation in which bad actors have either “more data” or “less data” (described further below) about each identity with which they are attempting to apply.

Neither misrepresentation rates nor types are necessarily consistent over the life of a program. Changes in the program’s internal controls may deter bad actors who deliberately misrepresent identities, which in turn, could reduce misrepresentation rates of a particular nature. The simulation does not generate different transactions in response to a reconfiguration of internal controls. Instead, by selecting to reconfigure the hypothetical program’s internal controls, simulation users can view how the control changes affect the simulated outcomes of the same attempted transactions.

Within the simulation, attempts to control misrepresented identity affect all program applicants. Program controls may incorrectly classify proper applicants as misrepresenting their identities. Thus, the misrepresentation rate meaningfully influences the costs and benefits of internal controls. If a program’s misrepresentation rate is low, for example, even very well-calibrated controls are likely to have more false positives than true positives and vice versa.

Types of Misrepresented Identities

As described in the report, program applicants with misrepresented identities are not homogenous with respect to their intentions or their responses to internal controls. For the purposes of this simulation, we define three general categories of misrepresented identity:

Intentional: Attempts using More Data

For the purposes of the simulation, misrepresentation attempts that use more extensive data may be associated with a deep level of knowledge about a specific person’s identity and private information. As such, identity verification controls in the simulation are less likely to deter misrepresentation attempts that have more complete data.

Intentional: Attempts using Less Data

For the purposes of the simulation, misrepresentation attempts that use less data may use a list of information to apply for benefits under the partially compromised identities. Those committing misrepresentation with less data may lack sufficient knowledge about an identity to overcome detailed control checks. Further, controls that assess whether the same computer is repeatedly accessing an application system may be an effective response in the simulation to combat misrepresentation of this nature (see Digital Footprinting). Within the simulation, identity verification controls are more likely to deter misrepresentation attempts that use less data. Relative to attempts that use more extensive data, those attempting to misrepresent identities using less data may be more likely to apply again shortly thereafter with a new identity; however, the simulation does not consider repeat attempts at identity misrepresentation by a single bad actor.

Unintentional

An applicant (or someone working on the applicant’s behalf) may enter information incorrectly with no intention to deceive the program. We do not simulate unintentional misrepresented identity in this program.

Mechanics in Our Model

The mechanics outlined below are a simplification for our model and only for illustrative purposes. While we have taken steps to ensure this illustration is reasonably plausible, they do not comprehensively represent the full range of real-world outcomes. As such, this model should not be used as a basis for decisions about real world programs.

  • Misrepresentation using "more data" is less susceptible to digital footprint scanning because individuals committing this misrepresentation will be seen by the system fewer times, although they may make mistakes like applying from a foreign country without masking their location.
  • Misrepresentation using "more data" is less susceptible to deterrence, because individuals committing thorough misrepresentation have invested more resources to acquire the identity information. In contrast, those committing misrepresentation with less data will abandon their efforts attempt in any control escalation scenario, in favor of trying to apply using a new identity.
  • Click here to show/hide technical details
    • Users may select a misrepresentation rate between 0% and 50% of program applicants, with a default of 15%.
    • Users may select a rate for the percentage of misrepresentation attempts using "less data" that is between 0 and 100%, with a default of 75%.
    • We assume digital footprinting will fail to flag misrepresentation attempts using "more data" 0.9% of the time, in contrast to 0.45% failure for misrepresentation that uses "less data".
    • We assume thorough misrepresentation will be deterred at a rate equal to the applicant burden score (see "How will the hypothetical program operate -> Control Tiers: Risk-based vs. Uniform" tab), generally a single-digit percent rate; thus, the majority will pursue their attempts until being rejected at the maximum control tier or receiving their improper payment.
(Click to read more in the technical whitepaper)

System Walkthrough

The following visualization shows either a risk-based or a uniform transaction management system, depending on the user selection. In addition to considering whether a risk-based or uniform transaction management system is appropriate for their particular program and context, organizations should consider how they might design and arrange controls within a system to most effectively meet their needs.

Risk-based System Diagram applicant_supplies_data Applicant-Supplied Data footprint A) System-Detected Data (via Digital Footprint) applicant_supplies_data->footprint applicant_completes_controls Applicant Completes Controls fail Fail Intermediate Tier applicant_completes_controls->fail pass Pass applicant_completes_controls->pass finalfail Fail Final Tier applicant_completes_controls->finalfail footprint_pass Not Flagged by Footprint footprint->footprint_pass footprint_fail Flagged by Footprint footprint->footprint_fail data_sharing B) Data Sharing predictive_modeling C) Predictive Risk Model data_sharing->predictive_modeling risk_tier_applied D) Control Tier Assignment predictive_modeling->risk_tier_applied risk_tier_increase Control Tier Increased risk_tier_increase->risk_tier_applied risk_tier_applied->applicant_completes_controls deterred Applicant Deterred by Controls risk_tier_applied->deterred fail->risk_tier_increase pay Payment Issued pass->pay footprint_pass->data_sharing exit Exit finalfail->exit footprint_fail->exit deterred->exit
Uniform System Diagram applicant_supplies_data Applicant-Supplied Data footprint A) System-Detected Data (via Digital Footprint) applicant_supplies_data->footprint applicant_completes_controls Applicant Completes Controls pass Pass applicant_completes_controls->pass finalfail Fail Uniform Control Tier applicant_completes_controls->finalfail footprint_pass Not Flagged by Footprint footprint->footprint_pass footprint_fail Flagged by Footprint footprint->footprint_fail controls_applied Uniform Controls Applied controls_applied->applicant_completes_controls deterred Applicant Deterred by Controls controls_applied->deterred pay Payment Issued pass->pay footprint_pass->controls_applied exit Exit deterred->exit finalfail->exit footprint_fail->exit

Applicant-Supplied Data

Within the simulation, the hypothetical program allows applicants to complete an online form that is used for identity verification. Programs, such as the hypothetical benefits program in the simulation, may be able to adjust the amount of information collected on the application, as well as the sensitivity of this information. Requiring sensitive information may deter some eligible recipients from completing applications. Additionally, eligible recipients may lack required documents to prove their eligibility/legitimacy. Other program requirements, such as notarized, hand-signed, and/or original documents, may also deter eligible program recipients. In addition to the deterrence factor, more complex applications generally require more administrative costs to process.

System-Detected Data

System-detected data, such as digital footprinting, analysis of typing speed, internet protocol (IP) address information, etc., can serve as another tool to help mitigate improper payments. These internal controls can help identify instances of potential misrepresented identity (purposeful or accidental) with limited burden on applicants.

  • The system can log information such as IP address, typing speed (and other typing patterns), device type, and/or browser used.
  • Use of system-detected data requires investment in technologies such as applicant analytics and associated databases.
  • In the hypothetical illustration, digital footprinting automatically removes flagged transactions from the application process before applying any other internal controls. This control could also be designed without automatically routing out flagged cases. Routing out flagged transactions may reduce the administrative cost of applying additional internal controls to transactions that the system predicts are improper at a specific stage, but could potentially also route out some proper applicants. Organizations should consider whether routing out transactions using various internal controls is appropriate in their own contexts.

Data Sharing

Linking an applicant to their records in other agencies’ systems can help the hypothetical proram to verify the applicant-supplied information. For example, with more complete data on these individuals, the hypothetical program may be able to more accurately differentiate between individuals who have the same name.

Predictive Risk Model

Predictive models can estimate the likelihood a transaction would be proper or improper, then compares this likelihood to the risk of a transaction. The model leverages information such as inconsistencies between…

  1. applicant-supplied data and system data (e.g., location of submission different than location of record) and;
  2. application data and other existing database information

…to predict the likelihood of misrepresented identity. The hypothetical program may enhance the model performance with data sharing. The additional data may supplement applicant-supplied data to reduce possible model errors, and may allow the hypothetical program to associate previously detected improper activity with an identity tied to pending transactions.

Control Tier Assignment

In the simulation, a uniform control system applies the same controls to every transaction. When using a risk-based, multi-tier control system, the hypothetical benefits program imposes a minimum baseline set of controls on each transaction based on its dollar value. The user-selected system components may serve as additional risk indicators. If a risk indicator is raised, the system will expose the transaction to a more rigorous set of initial controls, relative to the controls it would have been assigned using the dollar value risk baseline. The selected controls may require the designated applicant to meet certain requirements (in-person verification, physical fingerprinting, etc.) that verify their identity, and allow them to receive payment. If the applicant responds to and fails a set of controls, the transaction can be escalated to the next control tier, wherin the applicant must respond to new control checks to continue their transaction.

Outcome

  • Transactions that fail the control system are either removed from the process or are escalated to a higher Risk Tier.
  • Transactions that pass the control system result in payments to applicants.
  • Applicants who do not complete the controls the system assigns them are considered deterred and exit the process, regardless of whether they are proper/improper applicants or whether they would have passed these control checks.

Overview of Control Tier Structure: Uniform vs. Risk-based

The following italicized text is from the report.

Program offices can select from two strategies to verify identities—one that applies controls uniformly to all transactions and one that calibrates controls based on a gradation of risks. A program office may find either system ideal depending on its risk assessment.

A program office may decide to use a uniform transaction management system if all transactions are similar in risk, if it is not feasible to triage transactions by risk level, or if a program office is legally obligated to treat all transactions identically. A uniform system may be relatively easy to design and may effectively reduce improper payments but, depending on the controls the program office selects, may also cause broad burdens on both administrators and individuals not misrepresenting their identities because the system applies the same set of controls to all transactions regardless of risk.

A program office may decide to use a risk-based transaction management system if it experiences varying levels of risk in processing transactions or if a program office has sufficient data to accurately assess each transaction’s given risk level. This approach uses progressively more-rigorous tiers of controls, each of which corresponds to a given transaction’s risk level. In order to assess a transaction’s risk level, several panelists encouraged the use of predictive modeling.

For more information, please visit page 27 in the report.

Control Tier Structure in the Simulation

Simulation users can select whether the system routes all transactions to the same sets of controls (uniform transaction management) or to different sets of controls (risk-based transaction management) based on assessments of transaction risk.

By enabling the “advanced options” in the “How will the hypothetical program operate?” box, advanced users may also choose to modify the composition of controls in each tier.

Uniform (Single-Tier) Transaction Management System

As described above, exposing every applicant to the same system of controls can be effective at reducing improper payments and relatively easy to design, but may cause increased administrative costs as well as increased applicant burden (relative to the burden an applicant may experience in a risk-based system). For example, users may select “Uniform, High” if they determine that all transactions pose substantial risk, and want to uniformly apply the same relatively rigorous set of controls to all transactions. Yet, if a risk-based, multi-tier system would otherwise categorize transactions as low or medium risk, the “Uniform, High” selection will present applicants with controls that would be more burdensome than in the risk-based system. In the simulation, users may select from the first three levels of controls for a uniform, single-tier option. The highest level of controls is reserved for the riskiest transactions within the multi-tier system (described below).

Risk-based (Multi-Tier) Transaction Management System

As described above, applicants will pose different levels of risk to the hypothetical program – for example, because of the dollar value of their transaction. Further, the applicant-supplied and system-detected data can offer further insights into the level of risk.

A risk-based, multi-tier control system can also be effective at helping programs to mitigate improper payments. Within the simulation, more-rigorous controls are more likely to inconvenience or deter proper applicants and will consume more administrative resources than will less-rigorous controls. Thus, the multi-tier system design option only imposes more-rigorous controls in transactions assessed with higher risks.

The simulation uses a four-tier system of controls that incorporates a concept of escalating risk management with increasing levels of assessed risk of improper payment. Risk is a function of flags from mechanisms that authenticate identity and payment size (i.e., flags from selected system components and higher payment size equates to greater risk). This framework allows simulation users to modify the different levels of controls before running the simulation.

Mechanics in Our Model

The mechanics outlined below are a simplification for our model and only for illustrative purposes. While we have taken steps to ensure this illustration is reasonably plausible, they do not comprehensively represent the full range of real-world outcomes. As such, this model should not be used as a basis for decisions about real world programs.

Overview of Digital Footprint Monitoring

The following italicized text is from the report.

Digital footprint identifies information about an applicant’s hardware and software in order to connect past and future interactions with the same individual. This control may analyze items such as an applicant’s internet protocol (IP) address, time zone, language settings, etc. and would alert a program office if any of these features change. For example, if an application’s IP address is based out of another country, a program office may assess the transaction at a higher risk and require more stringent controls. Or, a program office reviews the applicant’s keystroke and/or mouse patterns to determine if the behavior suggests the application was completed by a bot.

For more information, please visit page 33 in the report.

Digital Footprint Monitoring in the Simulation

Simulation users can select whether the hypothetical program’s control system will try to identify the “digital footprint” of applicants, flagging those who appear to be attempting to use bots to apply using misreprsented identities.

Within the simulation, digital footprint monitoring has no applicant burden and a low administrative cost; hypothetical program applicants need only to interact with the system for footprinting to occur.

Mechanics in Our Model

The mechanics outlined below are a simplification for our model and only for illustrative purposes. While we have taken steps to ensure this illustration is reasonably plausible, they do not comprehensively represent the full range of real-world outcomes. As such, this model should not be used as a basis for decisions about real world programs.

  • In the simulator, a digital device footprint will only flag interactions that exhibit overtly questionable behavior, such as attempting to execute many transactions from the same IP and location, both of which are unrelated to the purported identities.
  • Click here to show/hide technical details
    • We assign a false positive rate of 2.5% , based on the notion that some proper applicants may share a connection, use a personal VPN, or otherwise inadvertently trigger this control.
    • We assign a false negative rate of 90% if the misrepresented identity uses "more data," implying that the applicant is engaged in a one-off activity. In the simulator, the digital footprint is less sensitive to these one-off activities.
    • We assign a false negative rate of 45% if the misrepresented identity uses "less data," implying that the applicant is attempting to use bots to apply multiple times using a list of different partial identities. This false negative rate reflects the fact that some identity thieves use sophisticated methods to randomize or obscure their digital fingerprint.
(Click to read more in the technical whitepaper)

Overview of Data Sharing, Eligibility Verification, and Do Not Pay

The following italicized text is from the report.

An authoritative source has access to information from an issuing source, so that relying parties such as agencies can validate evidence that an applicant provides. A program office may be more successful in verifying applicant identities if it uses multiple sources because data from the sources are often complementary and provide additional context for verification.

Several panelists noted that data sharing among government agencies and the private sector could increase the effectiveness of identity verification and reduce the likelihood of improper payments. Some panelists noted that data sharing can facilitate risk modeling, as more data can provide more accurate analytics.

For example, the Department of the Treasury’s Bureau of the Fiscal Service’s (Fiscal Service) Do Not Pay (DNP) business center collects data elements from a number of sources to assist participating agencies in determining an applicant’s payment eligibility. Incorporating these data, which include elements such as SSNs, known aliases, and business names, may help a program office identify anomalies. The program could then use network analysis to further identify potential bad actors.

Information hubs exist in the public sector (e.g., the DNP Business Center, the National Association of State Workforce Agencies’ integrity data hub, and the Financial Crimes Enforcement Network’s (FinCEN) suspicious activity repository) and private sector (e.g., MITRE’s aviation program), and can integrate either data or metadata from other sources. Information hubs rely on well-established, secure data pipelines to improve the efficiency of data sharing. They allow for established mechanisms to request and transfer data between agencies, which reduces the administrative burden for agencies, compared with initiating ad hoc data-sharing requests without a hub.

For more information, please visit page 30, 37, and 70 in the report.

Identity Verification Linked to Eligibility Verification in the Simulation

Simulation users can enable data sharing with DNP for cross-program analysis, which helps to uncover identities enrolled in multiple benefits programs, for whom there may be an eligibility issue. By screening program applicants during the pre-payment phase, the program can identify ineligible identities prior to making payments to ineligible applicants. The option to add data sharing for eligibility verification prior to executing transactions is available at no additional direct administrative cost to the hypothetical benefits program.

As described above, data sharing across additional information sources can enhance predictive models for identity verification by reducing false negatives and increasing true positives. Resolving an applicant’s identity across multiple federal programs can also minimize the costs of verifying certain eligibility standards.

The hypothetical benefits program makes payments only to living applicants, but does not have direct access to death records. One solution, noted in the report text above, is the DNP Business Center, a free service for Federal programs. DNP provides programs with access to screen payment recipients against multiple authorized data sources to determine eligibility. DNP also offers other analytics services to assist programs in their payment integrity activities.

Mechanics in Our Model

The mechanics outlined below are a simplification for our model and only for illustrative purposes. While we have taken steps to ensure this illustration is reasonably plausible, they do not comprehensively represent the full range of real-world outcomes. As such, this model should not be used as a basis for decisions about real world programs.

  • DNP's cross-government checks identify applicants with misrepresented identities as ineligible participants based on eligibility checks against the DNP death data sources, after identity resolution. The greater the access to data, the lower the error rates of predictive models.
  • Click here to show/hide technical details
    • Admin costs: $0 (free to utilize the DNP bulk matching service).
      • Note: The bulk matching service via the DNP Portal (data hub) has data format requirements. While customers may need to reformat some of their data, these costs are not reflected in the simulator design.
    • Control assignment, DNP true positive: automatically assigns 95% of misrepresented identities to the highest control tier.
    • Control assignment, DNP false positive: automatically assigns 0.5% of proper identities to the highest control tier.
    • Predictive model false positive rate: decreases by 50% of original rate.
    • Predictive model false negative rate: decreases by 50% of original rate.
(Click to read more in the technical whitepaper)

Overview of Predictive Models for Identity Verification

The following italicized text is from the report.

In order to assess a transaction’s risk level, several panelists encouraged the use of predictive modeling. The Office of Management and Budget (OMB) guidance notes that when establishing a robust data analytics program, using a predictive approach can move an agency from a “pay-and-chase” approach to one that allows the agency to identify the potential improper payments before they occur.

Predictive modeling is a class of statistical techniques whereby transactions that have preestablished criteria, patterns, or characteristics are associated with an estimated likelihood of being improper. Program offices can define and apply business rules based on management goals, such as the relative importance of avoiding false positive and false negative classifications. These rules, which can be automated, deem certain transactions higher risk, allowing a program office to control them appropriately during both pre- and postpayment processes.

For more information, please visit page 29 in the report.

Predictive Models in the Simulation

Simulation users can select whether the system uses no predictive model, a simple predictive model, or sophisticated predictive model to triage transactions within the risk-based transaction management process.

As described above, predictive models for identity verification estimate the likelihood that a pending transaction involves a misrepresented identity. Predictive models calculate these estimates based on information about the application as well as other information related to the program. In general, predictive models work best when previous data is representative of future data. Therefore, in the simulator, the predictive models function so that past transactions within the hypothetical benefit program are representative of the program’s future transactions.

In ideal circumstances for the use of a predictive model, applicants with one attribute always misrepresent their identity, and applicants without this attribute never misrepresent their identity. Yet, in reality, likelihoods are generally impacted only marginally for a single attribute.

As described above and expanded upon in the Metrics section, predictive models often must make a trade-off between false positive results (incorrectly labeling a true identity as false) and false negative results (incorrectly labeling a false identity as true). A false positive may lead to subjecting proper transactions to additional controls. Within the simulation, increasing the burden on applicants to complete additional controls, may deter them from completing the application process. On the other hand, a false negative may result in an improper payment to an ineligible recipient.

Mechanics in Our Model

The mechanics outlined below are a simplification for our model and only for illustrative purposes. While we have taken steps to ensure this illustration is reasonably plausible, they do not comprehensively represent the full range of real-world outcomes. As such, this model should not be used as a basis for decisions about real world programs

  • The simulation user may select no predictive model, a simple predictive model, or a sophisticated predictive model depending on the circumstances judged to be most appropriate.
  • A basic process with no predictive model assigns a risk tier (1-4) to applicants based on the size of their requested payment. For example, an applicant in the third quartile of payment size (50th to 75th percentile) will be placed in tier 3.
  • If either the simple or sophisticated model is used and an applicant is predicted to be misrepresenting their identity, that applicant's risk tier will be increased by 1 (to a maximum of 4) over the basic process.
  • Click here to show/hide technical details
    • A simple model is assumed to have a false positive rate of 5% and a false negative rate of 60%.
      • Note that if the program's rate of misrepresented identity is low enough, a majority of applicants predicted to be misrepresenting their identity may in fact be proper applicants.
    • A sophisticated model is assumed to have a false positive rate of 2.5% and a false negative rate of 30%.
(Click to read more in the technical whitepaper)

Overview of Control Tier Design

The following italicized text is from the report.

Remote verification allows an agency to verify an individual without physical access to a person or an item. A program office may conduct remote verification directly or rely on a third-party service, such as a CSP. Remote verification of a person (e.g., facial scan, or something a person is) or a document (e.g., driver's license, or something a person has) can occur. Remote verification can occur through various media, such as through the use of digital footprint monitoring, biometrics, bank-account verification, and physical-mail verification.

Remote verification offers applicants and agencies a relatively convenient solution, but the technology to circumvent these controls is constantly adapting. For example, in the event of verification by streaming video, some providers have implemented liveness detection to determine that an applicant is physically present rather than an inanimate artifact or injected data in order to combat attempts at impersonation of a person. A panelist noted that investing in an automated, remote identity-verification infrastructure may reduce costs by requiring less manual effort in the process and increase customer satisfaction, through the convenience of applying from home.

In-person verification requires an applicant to physically visit a location to verify identity. A program office may conduct this service on its own, or it may rely on a third party for this service. Once an applicant arrives at a location, a verifier may analyze the applicant's identity documents, gather the applicant's biometric data, or perform both practices. In-person verification is more costly than remote verification but may be necessary to provide a channel for those who are unable to verify their identities remotely. In addition, this method of identity proofing is generally considered to be a strong approach because it allows for direct physical comparison of an individual's documentation to the individual attempting to enroll.

For more information, please visit page 35 in the report.

Design of Control Tiers in the Simulation

The simulator contains four control tiers, each with two internal controls that the report discusses. The simulator groups the controls by a rough order of magnitude comparison of each control's respective cost, effectiveness, and applicant burden. The tiered approach and method of selecting the controls in each tier are for the purposes of the simulation, and the default settings may be sufficient for most users to visualize hypothetical system comparisons. If you would like to modify the controls in each tier, you may choose to remove one of the two controls that are selected by default. Before making your selections, please review information below about each control. The table below provides information about the relative effectiveness of available controls in identifying misrepresented identity; tendency to falsely identify misrepresented identity among proper applicants; administrative costs (burden on agency officials and resources); and burden on the applicant.

Information About Control Choices

Selected Control System Properties


The table below provides information about the baseline performance of the control scheme selected above.
(Click to read more in the technical whitepaper)

Overview of Funding

The following italicized text is from the report.

Costs to create and maintain identity-verification processes within a federated Identity, Credential, and Access Management (ICAM) framework may be substantial, considering direct procurement and deployment, administrative, and hard-to-measure costs, such as time, program reputation, and financial impact of fraudulent activity. Many program offices face a direct trade-off between controlling risks related to a lack of identity verification and effectively achieving goals, such as timely issuing benefits to the correct individuals. A successful identity-verification framework cost effectively improves identity determinations while avoiding undue burden on affected individuals and helps to prevent and reduce improper payments.

Centralizing the funding for implementing a federated identity framework may ensure long-term stability. Some experts favored centrally funding through the federal government because they view it as the federal government’s responsibility to effectively deliver benefits to the public, and noted that the way to do that is via effective identity solutions. One example of this type of fund is the Technology Modernization Fund, which certain agencies may use to improve, retire, or replace existing federal information technology systems. An expert noted that establishing a similar type of fund to allow program offices to invest in identity-verification processes would make more sense than relying on each agency’s own appropriated funding. A dedicated source of capital that provides program offices flexibility to invest in new identity-verification projects could generate cost savings and may reduce improper payments.

For more information, please visit page 55 and 64 in the report.

Total Administrative Costs in the Simulation

Simulation users can select a budgeted amount for the hypothetical benefits program as well as whether to apply for additional funding.

Within the simulation, the total cost of selected internal controls is visible on the “Can you improve the hypothetical program’s performance?” tab and in the corresponding summary panel under “System Settings”. After selecting the program’s budget, different combinations of internal controls will result in the hypothetical program’s being over- or under-budget. The hypothetical costs associated with each system design choice are explained on the corresponding configuration page, under the “How will the hypothetical program operate?” tab.

It is important that internal controls are cost-effective because programs’ budgets are limited, and the simulation user must assess the benefits of the implemented internal controls relative to their costs. Federal agencies may have resources available to them to help improve their payment integrity. As mentioned above, an example of these additional resources available to agencies as-of the launch of this tool is the Technology Modernization Fund (TMF), which was established to provide transfers to certain agencies that apply for help to improve, retire, or replace existing information technology systems. (TMF transfers also generally require the recipient agency to pay reimbursement and fees; however, for simplicity, the simulation tool does not reflect such requirements.)

Users of the simulation tool can select the option to “apply” for additional funding for their hypothetical benefits program. If users select this option, the simulation automatically adds funds are the budget, which must be spent on identity verification controls. For the purposes of the simulation, the hypothetical program incurs no additional adminstrative costs to select this option.

Mechanics in Our Model

The mechanics outlined below are a simplification for our model and only for illustrative purposes. While we have taken steps to ensure this illustration is reasonably plausible, they do not comprehensively represent the full range of real-world outcomes. As such, this model should not be used as a basis for decisions about real world programs.

  • Applying for additional funding will result in a 10% increase in available funds for the hypothetical program.
(Click to read more in the technical whitepaper)

Overview of Security

The following italicized text is from the report.

The E-Government Act of 2002 requires federal agencies to conduct privacy impact assessments of privacy risks associated with information technology used to process personal information. Similarly, The Office of Management and Budget (OMB) specified actions that agencies should take to prevent data breaches, such as using encryption to protect personally identifiable information (PII).

For program offices to achieve a certain level of identity assurance, the public has to provide some identity attributes as part of the verification process. An important consideration regarding sharing personal information among government programs is whether it can be done without sacrificing an individual’s right to privacy.

For more information, please visit page 47 in the report.

Security in the Simulation

“Over 28,000 security incidents were reported by federal civilian agencies to the Department of Homeland Security in Fiscal Year 2019. Since many government Information Technology (IT) systems contain vast amounts of PII, federal agencies must protect the confidentiality, integrity, and availability of this information &#8212 and effectively respond to data breaches and security incidents. Likewise, the trend in the private sector of collecting extensive and detailed information about individuals needs appropriate limits.” (https://www.gao.gov/cybersecurity)

The simulation incorporates these real-life cyber threats. To simulate these risks, each simulated program participant has a chance that their data/identity is stolen via a security breach. To address this risk, simulation users must decide if they want to dedicate a portion of their hypothetical budget toward enhancing IT/system security. If so, the simulation allows users to select two levels of additional security options beyond the minimum viable security to operate the hypothetical benefits program.

Description of Options

Users can also decide to apply cybersecurity measures to their program to reduce the likelihood that their participants’ data is stolen by hackers and simulate those effects.

The default security setting for the simulation is “Minimal Viable Security”. Under this setting there are no added security measures in place to reduce the chance that a participant is the victim of a hacking event. In turn, there are no additional administrative costs for the program. Users can choose this default setting to simulate the outcomes of not implementing any enhanced security features.

For the purposes of the simulation, the costs between the security options vary significantly. There is no set administrative cost for implementing the security. Instead, the costs of implementing the two security options are based on the implemented internal controls, as any security measure in the simulation must be robustly applied to each of the implemented internal controls.

Note: the simulation does not allow users to simulate a benefits program that is impervious to cyberthreats.

Assumptions in Our Model:

The mechanics outlined below are a simplification for our model and only for illustrative purposes. While we have taken steps to ensure this illustration is reasonably plausible, they do not comprehensively represent the full range of real-world outcomes. As such, this model should not be used as a basis for decisions about real world programs.

  • If a simulated program participant is associated with a security breach, all other participants whose application happens on the same day or any day leading up to the security breach is assumed to have their data compromised.
  • Click here to show/hide technical details
    • A simulated program participant's chance of being associated with a security breach victim is 0.2% per participant.
    • Applying the "Enhanced" security measures decreases a participant's chance of being a security breach victim by 30%. As a result of the increased security measures, costs also increase by 30%.
    • Applying the "Further Enhanced" security measures decreases a participant's chance of being a security breach victim by 60%. As a result of the increased security measures, costs also increase by 60%.
(Click to read more in the technical whitepaper)





This simulation tool is an illustrative tool that uses hypothetical data and provides hypothetical outcomes based on the parameters that users select. This simulation is not an optimization tool, and the outcomes are not generalizable. We have not assessed the practices implemented in this simulation tool to determine whether agencies could implement them under their existing legal framework, nor if this would require changes in this legal framework.

The Joint Financial Management Improvement Program (JFMIP) is a cooperative venture between the Department of Treasury, the Office of Management and Budget (OMB), the Office of Personal Management (OPM), and the U.S. Government Accountability Office (GAO).
Part of the
GAO Science, Technology Assessment & Analytics Team
GAO Contact:
Taka Ariga, Chief Data Scientist: arigat@gao.gov
Acknowledgments:
Alex Gromadzki, Senior Data Scientist
Andrew Kurtzman, Assistant Director