Do Standardized Test Makers Use AI to Help Write Test Questions?

AI as a Drafting Aid Within a Human-Led Process

Some standardized testing organizations have begun incorporating AI into parts of their test development process, particularly as a tool to help generate draft candidate questions more efficiently. Writing a high-quality test question — one that’s clear, unambiguous, appropriately difficult, free of unintended bias, and genuinely aligned with the specific skill or knowledge it’s meant to assess — is a skilled, time-consuming task when done entirely manually by trained item writers. AI-assisted drafting can help generate a larger initial pool of candidate questions more quickly, giving human test developers more raw material to work with and refine.

This is a meaningfully different role than “AI writes the test,” though — the drafting assistance is generally one early step within a larger, still human-led development and validation pipeline, not a replacement for the expertise and judgment involved in finalizing what actually appears on a real test.

Why Human Review Remains Essential

Test questions used in high-stakes standardized assessments go through rigorous review processes designed to catch exactly the kinds of problems that could undermine a test’s fairness or validity: ambiguous wording that could be interpreted multiple ways, unintended bias that might disadvantage certain groups of test-takers, or a question that doesn’t actually measure the skill it’s supposed to measure. AI-generated draft questions are not exempt from needing this same level of scrutiny — if anything, given the well-documented tendency of AI systems to occasionally produce plausible-looking but flawed or inaccurate content, human review of AI-assisted questions is treated as an essential, non-negotiable step rather than an optional check.

Professional test development organizations generally maintain established quality-control processes, including expert review panels and statistical analysis of how questions perform once used, and these processes apply to AI-assisted questions just as they would to any other candidate question before it becomes part of an actual, scored test.

Why Practices Vary and Aren’t Always Fully Public

The specific extent to which any given testing organization uses AI in question development, and exactly how that process is structured, isn’t uniformly public or standardized across the industry. Different organizations may be at different stages of exploring or adopting these tools, and the details of proprietary test development processes aren’t always disclosed in full. This means general statements about “AI in test development” should be understood as describing an emerging trend across the field, not a specific, verified description of any single test’s exact process.

Bottom Line

Some standardized test makers do use AI to help draft candidate test questions, primarily as a way to speed up the early stages of item generation, but rigorous human review, revision, and validation remain a standard, essential part of finalizing any question before it’s used in an actual test — meaning AI currently functions as a drafting aid within a human-led process rather than a replacement for expert test development.

Go deeper

• What was the Turing Test originally meant to prove?
• What’s a Good AI Prompt for Writing Ad Copy Variations to A/B Test?
• What is a hallucination rate and how do researchers actually measure it?