How do government agencies audit AI systems for bias after deployment

Monitoring Real-World Outcomes, Not Just Initial Testing

Government agencies audit deployed AI systems for bias primarily by analyzing real-world outcome data across different demographic groups for statistically significant disparities, recognizing that bias can sometimes emerge or become more apparent once a system is used at scale on real, diverse populations, beyond what pre-deployment testing alone might reveal.

Analyzing Outcome Data Across Demographic Groups

A core auditing practice involves systematically comparing actual outcomes — approval rates, error rates, or other relevant measures — across different demographic groups to check for statistically significant disparities that might indicate the system is producing meaningfully different results for similarly situated individuals based on protected characteristics.

Reviewing Complaint and Appeal Patterns

Beyond direct outcome data analysis, reviewing patterns in citizen complaints and formal appeals can help surface potential bias issues, since a disproportionate volume of complaints or successful appeals from a particular demographic group can be an important signal worth investigating further, even when it’s not immediately obvious from aggregate outcome statistics alone.

The Role of Independent Third-Party Audits

Some agencies commission independent third-party reviews of their AI systems, bringing in outside technical or civil rights experts to conduct a more independent evaluation than internal agency review alone might provide, adding a valuable additional layer of scrutiny, particularly for higher-stakes or more controversial AI use cases.

Why Consistency and Rigor Vary Considerably Across Agencies

Despite these available practices, the actual frequency, rigor, and consistency of post-deployment bias auditing varies considerably across different government agencies and programs, since this kind of ongoing monitoring isn’t yet uniformly mandated or standardized, meaning some agencies conduct considerably more thorough and regular bias audits than others.

What Typically Happens When an Audit Finds a Significant Problem

When a bias audit reveals a significant disparity, appropriate responses generally depend on the specific situation and severity, and can include retraining or adjusting the underlying system, adding additional human review safeguards for affected decision types, or in more serious cases, pausing the system’s use until the identified issue is adequately addressed and resolved.

Why This Remains an Actively Evolving Area of Government AI Governance

Given documented inconsistency in current bias auditing practices, this remains an area of active policy attention, with ongoing advocacy and proposed guidance aimed at establishing more consistent, mandatory post-deployment monitoring requirements across government agencies, rather than relying on inconsistent, largely voluntary current practice.

Bottom Line

Government agencies audit deployed AI systems for bias by analyzing real-world outcome disparities across demographic groups, reviewing complaint and appeal patterns, and in some cases commissioning independent third-party reviews — though the consistency and rigor of this kind of post-deployment auditing varies considerably across agencies, since it isn’t yet uniformly mandated or standardized government-wide.

Go deeper

• How Do Companies Test AI Models for Bias Before Release?
• Can AI Bias Be Completely Eliminated?
• What Real-World Harms Have Resulted From Biased AI Systems?