On Oct. 30, President Biden issued an executive order on the safe, secure, and trustworthy development and deployment of artificial intelligence (AI). For reference, see our article on the executive order. Not surprisingly, the National Institute of Standards and Technology (NIST) was one of the first agencies to respond to the executive order. As is evidenced by previous NIST frameworks, NIST guidelines impact both the public and private sectors across various industries.
NIST’s first published response acknowledges its obligations from the executive order to supplement and develop new guidelines and standards for AI testing and development. NIST's comprehensive response is here, but below is a summary of its proposed initiatives:
- Develop a companion resource to the AI Risk Management Framework focused on generative AI.
- Develop a companion resource to the Secure Software Development Framework to incorporate secure-development practices for generative AI and dual-use foundation models.
- Launch a new initiative to create guidance and benchmarks for evaluating and auditing AI capabilities, with a focus on capabilities that could cause harm.
- Establish guidelines and processes – except for AI used as a component of a national security system – to enable developers of generative AI, especially dual-use foundation models, to conduct AI red-teaming tests for deployment of safe, secure, and trustworthy systems.
- Engage with industry and relevant stakeholders to develop and refine guides and best practices.
- Develop a report identifying existing standards, tools, methods, and practices, as well as the potential development of further science-backed standards and techniques, for authenticating content and identifying synthetic content.
- Create guidelines for agencies to evaluate the efficacy of differential-privacy-guarantee protections, including for AI.
- Develop guidelines, tools, and practices to support agencies' implementation of minimum risk-management practices.
- Assist the Secretary of Commerce in coordinating with key international partners and standards development organizations to drive the development and implementation of AI-related consensus standards, cooperation, and information sharing.
Joseph "Joe" Damon, Leslie Green, Jackson Parese, and Marc Jenkins contributed to this article.