The US Food and Drug Administration (FDA) published an Action Plan for artificial intelligence (AI) and machine learning (ML) software on January 12, 2021 that provides near-term actions to develop a regulatory framework for AI and ML-based medical devices.  The quick takeaway is that FDA will publish a draft guidance on change control plans, a key concept from its April 2019 discussion paper on AI/ML-based software devices (previously reported here). FDA also will hold a public workshop on algorithm transparency and engage its partners and stakeholders on other key initiatives, such as evaluating bias in algorithms. While the Action Plan proposes a roadmap for advancing a regulatory framework, an operational framework appears to be further down the road.

Highlights of the Plan

1. FDA Will Host a Public Workshop on Algorithm Transparency

The FDA plans to hold a public workshop to elicit input from stakeholders on end user transparency. Specifically, FDA will seek input on the types of information that manufacturers should include in the labeling of AI/ML-based medical devices to ensure that end users can understand the benefits and risks of the device. FDA did not provide a timeline for the workshop, but the agency presumably is contemplating 2021.

2. Evaluating Algorithm Bias

Much has been written about the potential bias in AI/ML systems due to the historical data sets used to train the AI algorithm. According to FDA’s plan, the agency will continue to work with its research partners, including renown universities, to develop a methodology for identifying and evaluating bias—such as race and ethnicity—in AI/ML algorithms.

3. FDA Will Publish Draft Guidance on Change Control Plan in 2021

In a more concrete action, FDA committed to publishing a draft guidance document in 2021 on the “Predetermined Change Control Plan” described in its April 2019 white paper, “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) – Discussion Paper and Request for Feedback” (the “Discussion Paper”). The Discussion Paper proposed a framework for modifications to AI/ML-based software on the principle of a Predetermined Change Control Plan.

A Predetermined Change Control Plan includes the types of anticipated modifications, and the associated methodology used to implement those changes in a controlled manner—in other words, what aspects are intended to change through the algorithm learning, and how the algorithm will learn and change, based on the inputs. The draft guidance will include recommendations on what should be included in the Predetermined Change Control Plan during a device’s premarket review process.

4. Pilot Program for Real-World Performance Monitoring

The April 2019 Discussion Paper described the role of real-world data in monitoring the ongoing safety and effectiveness of an AI/ML-based device. Monitoring real-world data (i.e., data gathered on the device’s performance post launch), and leveraging that data to inform product changes and evaluate algorithm behavior, is a key concept to a total product lifecycle (TPLC) approach adopted by the Agency.

Under the Action Plan, the FDA will pilot a voluntary program aimed at eventually developing a  framework that can be used for gathering and validating real world parameters and metrics for AI/ML-based devices. FDA did not provide a timeline for a pilot program.

5. Building Consensus for Good Machine Learning Practice (GMLP)

The Discussion Paper used the term “Good Machine Learning Practice” (GMLP), to describe AI/ML best practices (e.g., data management and relevance, algorithm training, validation and documentation) that are similar to good software engineering practices or quality system practices. Under the Action Plan, FDA will continue to participate in various global working groups that are focused on developing and harmonizing principles of GMLP.

What’s Next for Developers of AI

To developers of AI/ML, the Action Plan may appear modest in its objectives for 2021. For example, the only specific commitment for 2021 is to publish a draft guidance on Predetermined Change Control Plans, which is only one aspect of the Agency’s multi-pronged approach laid out in its Discussion Paper. Developers, however, can view this as a continued opportunity to engage the FDA and influence the agency’s thinking on key concepts that will eventually be incorporated into a comprehensive framework.