In an era where artificial intelligence (AI) is reshaping landscapes in the healthcare industry and beyond, understanding the governance of AI technologies is paramount for organizations seeking to utilize AI systems and tools. AI governance encompasses the policies, practices, and frameworks that guide the responsible development, deployment, and operation of AI systems and tools within an organization. By adhering to established governance principles and frameworks, organizations can ensure their AI initiatives align with ethical standards and applicable law, respect human rights, and contribute positively to society. Various international organizations have set forth AI governance principles that provide organizations with a solid foundation to develop organizational AI governance based on widely shared values and goals.
Embracing OECD AI Principles for Responsible Stewardship
The Organisation for Economic Co-operation and Development (OECD) set a landmark in May 2019 by adopting a comprehensive set of AI Principles to promote the responsible stewardship of AI in a manner that is human-centered and trustworthy. The OECD’s AI Principles aim to foster innovation and economic growth while ensuring AI systems are developed and deployed in accordance with human-centered values, including respect for human rights, democracy and economic growth. The OECD AI Principles emphasize:
- Inclusive Growth, Sustainable Development, and Well-Being: AI should contribute to economic advancement and global well-being, without sacrificing future generations’ ability to meet their needs.
- Human-Centered Values and Fairness: AI systems must respect the rule of law, human rights, democratic and human-centered values. This includes, for example, non-discrimination and equality, privacy and data protection, and fairness. This also includes addressing misinformation and disinformation amplified by AI, while respecting freedom of expression and other rights.
- Transparency and Explainability: Stakeholders should understand AI systems’ workings and outcomes. This includes understanding and being aware of their interactions with AI systems, having access to plain and easy-to-understand information on the inputs and operation of the AI system to understand its output, and having sufficient information to challenge an AI system’s information if a stakeholder is adversely affected by it.
- Robustness, Security, and Safety: AI systems should be robust, secure and safe so that they function appropriately in all conditions and do not pose unreasonable safety and security risks. Mechanisms should be in place to protect against and mitigate such risks, including mechanisms to bolster information integrity.
- Accountability: Those involved in the AI system lifecycle should be accountable for its functioning in line with the above principles. AI actors should ensure traceability with regard to AI systems, including in relation to datasets, processes and decisions, to enable analysis of the AI system’s outputs. Organizations should apply a systematic risk management approach to each phase of the AI system lifecycle on an ongoing basis.
Following these principles, the OECD makes the following recommendations for policymakers:
- Investment in AI R&D: Encouraging public and private investment in AI research, including interdisciplinary efforts and open datasets.
- Fostering an Inclusive AI Ecosystem: Developing a dynamic, sustainable, and interoperable digital ecosystem for trustworthy AI.
- Enabling Governance and Policy Environment: Promoting an agile policy environment that supports AI from R&D to deployment and operation, with outcome-based approaches that provide flexibility in achieving governance objectives.
- Building Human Capacity and Preparing for Labor Market Transformation: Equipping people with necessary skills for AI and ensuring a fair transition for workers, including empowering people to effectively use and interact with AI systems.
- International Cooperation: Advancing these principles globally through cooperation in the OECD and other fora, promoting global technical standards for trustworthy AI.
Fair Information Practice Principles: The Backbone of Privacy in AI
As AI technologies collect, process, and utilize personal information, the need to protect individual privacy while ensuring the integrity and utility of these systems becomes increasingly important. This is where the Fair Information Practice Principles (FIPPs) come into play, providing a foundational framework for responsible information management. The FIPPs, rooted in a 1973 Federal Government report and having informed both federal statutes and global privacy policies, offer a robust guide for businesses and in-house counsel navigating the complex interplay of AI, privacy, and data protection.
- Access and Amendment: Ensuring individuals have appropriate access to their personal information (PII) and the opportunity to correct or amend it is crucial. In healthcare AI, where data accuracy can directly impact patient care and outcomes, the principle of access and amendment not only protects privacy but also underpins the reliability of AI-driven decisions.
- Accountability: Organizations must be accountable for adhering to privacy principles and applicable privacy requirements, necessitating clear roles, responsibilities, and compliance mechanisms. For healthcare entities utilizing AI, establishing rigorous audit trails and training programs is essential to maintain trust and accountability in AI systems’ outputs.
- Authority: Collecting, using, and disclosing PII must be grounded in legitimate authority to collect and process such PII. Healthcare AI applications, therefore, must be transparent about their legal basis for data handling, ensuring that all data processing activities are justifiable and documented.
- Minimization: Data minimization ensures that only necessary PII is collected and retained. In the context of healthcare AI, this principle advocates for the lean processing of data, mitigating risks of unauthorized access or data breaches while preserving the system’s efficiency and effectiveness.
- Quality and Integrity: The accuracy, relevance, timeliness, and completeness of PII are fundamental. Healthcare AI systems must incorporate mechanisms to ensure data integrity, as the quality of input data directly affects the quality of patient care and operational decisions.
- Individual Participation: Facilitating meaningful individual participation in the handling of their PII fosters transparency and trust. In healthcare AI, this could involve consent frameworks, privacy notices, and channels for privacy-related inquiries and complaints, promoting patient engagement and trust.
- Purpose Specification and Use Limitation: Defining clear purposes for PII collection and restricting data use to those purposes are critical. Healthcare AI initiatives must communicate their objectives transparently and ensure that data is not repurposed without a valid, legal, and communicated reason, safeguarding against misuse and maintaining public trust.
- Security: Robust security measures proportional to the risk and potential harm from data breaches are non-negotiable. Healthcare AI systems, given their sensitivity and the potential impact of data compromise, require state-of-the-art safeguards to protect PII against unauthorized or malicious activities.
- Transparency: Openness about PII policies and practices is essential for accountability and trust. Healthcare AI operations should provide clear, accessible information on their data handling practices, enabling patients and stakeholders to understand and engage with the AI systems confidently.
UNESCO’s Ethical Framework for AI
Adding to the global dialogue, the United Nations Educational, Scientific and Cultural Organization’s (UNESCO’s) recommendations on the ethics of AI set forth a comprehensive framework aimed at guiding ethical AI development and governance. Adopted in November 2021, these recommendations highlight the importance of embedding human rights, inclusion, transparency, and accountability within AI systems, providing a global consensus on ethical AI practices.
- Human Rights and Inclusion at the Core: Central to UNESCO’s recommendations is the commitment to human rights and inclusion, ensuring AI technologies respect every individual’s dignity and rights. Organizations must conduct ethical impact assessments to prevent biases and ensure AI’s accessibility and fairness, particularly towards marginalized groups.
- Transparency and Accountability: Transparency and accountability are vital for public trust in AI. In healthcare, this means making AI decision-making processes understandable and ensuring systems are fair and ethical. Organizations must prioritize explainability and establish governance structures for accountability and error correction.
- Sustainability: Sustainable AI involves considering its environmental, social, and economic impacts. For healthcare, this means AI systems that are energy-efficient, reduce waste, and contribute positively to society and the environment, aligning with the broader goals of sustainability and equity.
- A Call to Action for Ethical Leadership: The UNESCO recommendations present a call to action for ethical leadership in the development and application of AI in healthcare. Within an organization, this involves fostering a culture of ethical awareness and responsibility, engaging in multi-stakeholder dialogues to address ethical challenges collaboratively, and advocating for policies and practices that align with the highest ethical standards.
Operationalizing AI Principles through AI Governance Frameworks
While principles provide a foundation, AI governance frameworks offer the necessary guidance for operationalizing these values within organizations and they incorporate some of the principles discussed above. Examples of frameworks to which organizations may refer include: ISO 42001 (Management Systems) and ISA 31000:2018 (Risk Management – Guidelines); NIST AI Risk Management Framework; Institute of Electrical and Electronics Engineers 7000-21 Standard Model Process for Addressing Ethical Concerns during System Design; Human Rights, Democracy and Rule of Law Assurance Framework for AI Systems – Council of Europe; and other standards specific to particular industries and jurisdictions. These frameworks are not one-size-fits-all; they must be tailored to fit the specific needs and contexts of each organization, taking into account industry-specific requirements and applicable laws.
Conclusion
Understanding and implementing AI governance principles and frameworks is crucial for healthcare organizations seeking to leverage AI systems and tools in their operations. These guidelines and principles are intended not only to ensure compliance with ethical standards and legal requirements, but also to build trust with users and society at large by demonstrating a commitment to responsible AI development. As AI continues to evolve, staying informed and engaged with these governance principles and practices will be key to harnessing the technology’s potential while safeguarding fundamental values and rights and remaining compliant with the developing body of law related to AI.
Copyright © 2025, Sheppard Mullin Richter & Hampton LLP.