FDA Proposes Clinician-Style Competency Testing for Medical Generative AI
The Food and Drug Administration has released a discussion paper outlining a clinician-inspired, competency-based framework to evaluate medical generative AI devices before market entry. The proposed model signals a major shift in how federal agencies assess adaptive software in mission-critical environments.
The U.S. Food and Drug Administration (FDA) has unveiled a discussion draft proposing a "competency-based" evaluation framework for generative AI (GenAI) marketed as medical devices. Inspired by human clinical licensing exams, the proposed approach evaluates end-use, user-facing GenAI systems rather than foundational models or subcomponents. Under this model, regulators would assess AI capabilities across clinical knowledge, analytical reasoning, communication, safety behavior, and generalizability, using scoring rubrics and expert adjudication to determine premarket authorization safety and efficacy.
Acknowledging the rapid evolution and non-deterministic nature of large language models, the FDA’s proposed testing rigor is proportional to the device’s risk profile. Evaluators will measure whether an AI tool directs clinical decisions versus providing background contextual information, as well as the potential severity of faulty outputs. Because static premarket testing may prove insufficient for continuously updating systems, the agency is also considering mandatory post-market real-world evidence collection. Public feedback on the discussion draft is open through October 19, serving as a precursor to formal policy development.
Signal Pro feature
This signal is in the subscriber archive.
Signals older than seven days are reserved for members. Subscribe to read the full analysis and unlock the complete Defense Signals archive.
Executive takeaway
Subscribe to receive signals like this — plus the executive analysis behind them.
Get the Signals