The emergence of generative machine learning models, such as ChatGPT, has led to a surge in interest in artificial intelligence (“AI”) over the past year. This increased interest extends to the health care industry, where AI has the potential to dramatically transform the health care reimbursement and delivery system and accelerate health care innovations. Nevertheless, the use of AI is not without concern and needs to be balanced against due consideration of the risks. As questions related to the technology’s safety rise, bipartisan efforts are taking place to ensure federal agencies optimize the development and use of AI while working to address potential inherent risks. These efforts include promoting transparency and notice, ensuring fairness and non-discriminatory practices, and protecting the privacy and security of health information, which we previously wrote about here.

In furtherance of these efforts, on October 30, 2023, the Biden Administration introduced an eagerly-awaited Executive Order (EO) on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. The EO aims to promote AI innovation while protecting against potential harmful consequences, and specifically addresses the risks associated with the development and use of AI in health care. The EO seeks to allow the safe implementation of AI-enabled technologies in health care delivery, including with regard to quality measurement, performance improvement, program integrity, benefits administration, and patient experience, with the caveat that all such uses should incorporate an appropriate level of human oversight.

While the use of AI in health care is well underway, stakeholders are eager to understand how to effectively use AI while adhering to guardrails that may be, in the near future, mandated by state and federal legislatures. Below are several key takeaways and important timelines from the EO that health care stakeholders should be aware, and which provide opportunity for stakeholder engagement on AI issues that may impact the health care sector.

Key Takeaways

• Section 8 of the EO focuses on the risks and developments associated with AI in the health care industry, specifically covering critical areas including AI use in drug development, predictive/diagnostic AI use cases, safety, healthcare delivery and financing, and documentation and reporting requirements.
• The EO directs HHS to create an AI Task Force, an AI Assurance Policy, and an AI Safety Program. As a result of these directives, it is likely that HHS will collaborate with several other key agencies including the Office of the National Coordinator for Health Information Technology (ONC), the Centers for Medicare and Medicaid Services (CMS), and the Office for Civil Rights (OCR), among others.¹
• The EO does not currently affect regulations, rules or reporting requirements. The EO itself does not set any legal requirements related to the use of AI in the health care sector or other sectors. However, it sets the stage for the creation of new rules and regulations, particularly related to health care, in the near future.
• Long standing privacy and security rules continue to apply to the use of AI in health care. Companies should continue to account for and comply with existing privacy and security laws, such as HIPAA, HITECH, and the Federal Trade Commission (FTC) Act, among others, when their AI systems handle, create, receive, transmit or maintain data, including protected health information (PHI) and personally identifiable information (PII).

Key Timelines

Health-related frameworks and safety mechanisms must be established in accordance with the following timeline, which runs from October 30, 2023, the date the EO was released:

• Within 60 Days – HHS must appoint a Chief AI Officer who will be responsible for promoting AI innovation within HHS while managing risks associated with the use of AI.
• Within 90 Days – HHS must establish an AI Task Force to set forth a framework for “responsible use” of AI in the healthcare sector.
• Within 180 days – HHS must direct its component agencies to develop a strategy to determine whether AI-enabled technologies comply with the strategic plan laid out in Section 8(b)(i) of the EO (summarized in Appendix A to this alert).
• Within 180 Days – HHS must establish an AI Assurance Policy which incorporates pre-market assessment and post-market oversight of AI-enabled health care technologies.
• Within 180 Days – HHS must “consider appropriate actions” to ensure that health care providers receiving federal financial assistance comply with federal nondiscrimination laws.
• Within 365 days – HHS must establish a strategy for regulating AI use in drug development.
• Within 365 days – HHS, in consultation with the Secretary of Defense and the Secretary of Veterans Affairs, must establish an AI safety program in partnership with voluntary federally listed Patient Safety Organizations.
• The EO also requires HHS to create incentives under its grantmaking authority to promote/encourage responsible AI development and use. This will likely involve collaboration with private sector entities. The EO does not specify a deadline for this activity.

Although the EO does not create new legal requirements, it requires HHS and its component agencies to take specified regulatory actions. Therefore, stakeholders should expect to see more concrete rules and guidance from agencies soon. At this stage, the EO requires HHS to focus on procedural requirements around the use of AI (such as developing security systems, safety frameworks, and documentation methods), rather than substantive rules (such as prohibitions on implementing AI in certain use cases).

Implementation Guidance from the Office of Management and Budget (OMB): Soon after the EO was announced, the OMB released a Proposed Memorandum for the Heads of Executive Departments and Agencies (the “OMB Memo”). If finalized, the OMB Memo would require most federal agencies to designate a Chief AI Officer and develop an AI strategy that meets certain requirements. Agencies would also be required to submit compliance plans that ensure operations are consistent with the OMB Memo and make publicly available annual AI use case inventories. The OMB Memo emphasizes that agencies must ensure that their IT infrastructure, data management systems, workforce and cybersecurity platforms are robust enough to support AI applications. Finally, agencies would be required to terminate use of any noncompliant AI applications by August 1, 2024. OMB is accepting comments on this draft here until December 5, 2023.

The OMB Memo requires added scrutiny over AI use cases that are “safety-impacting” or “rights-impacting,” including for example, medical transportation, delivery of biological or chemical agents, prescription drug-related activity, decisions involving medical devices utilization, clinical diagnostic tools, health risk assessments, and interventions related to mental health care. For these use cases, an agency would be required to implement safeguards including a formal AI impact assessment, real world testing, independent evaluation of the AI tool, and processes for ongoing monitoring and mitigation of risks. For rights-impacting use cases, the OMB Memo also provides standards for detecting algorithmic bias, addressing disparate impacts, creating representative data sets, and obtaining feedback from affected groups. Stakeholders should review these lists closely, while also keeping in mind that the lists are not exhaustive and do not account for future or new AI use cases. 

In line with the Biden administration’s EO and the corresponding OMB Memo, the FDA has also been particularly active in paving the way for AI regulation and development. For example, the FDA previously published Discussion Papers discussing AI use in drug development released in May 2023. The FDA has also released information about regulating AI products as medical devices.

Industry stakeholders should be on the lookout for regulatory and policy frameworks, updates, and guidance in the coming months, especially from HHS and federal agencies that operate under HHS.

APPENDIX A

Section 8(b)(i) of the EO states that the HHS AI Task Force shall develop a strategic plan on responsible deployment of AI that covers the following areas:

• Development, maintenance, and use of predictive and generative AI-enabled technologies in healthcare delivery and financing
• Long-term safety monitoring of AI technology in the health sector
• Incorporation of equity principles to combat biases and unwanted discrimination
• Incorporation of safety, privacy and security standards to protect PII
• Documentation to help users determine safe uses of AI in local settings
• Plans to advance positive use cases and promote best practices in State, local Tribal and territorial settings
• Identification of AI uses that promote workplace efficiency and satisfaction