From Principle to Practice: Applying the FDA and EMA AI Guiding Principles in Clinical Trials
How Just in Time GCP Operationalizes the FDA and EMA Guiding Principles for Good AI Practice
By Kathie Clark, Industry Expert; Technology & Innovation Partner, Just in Time GCP
As AI-enabled tools are increasingly adopted across clinical operations, Life Science organizations must evaluate technologies that behave very differently from traditional software. In a recent post, we outlined the hard questions organizations should ask AI vendors to assess alignment with the Guiding Principles of Good AI Practice, jointly published by the FDA and the EMA. This article takes the next step by describing how organizations operationalize those expectations in practice. Specifically, it shows how teams apply the FDA and EMA Guiding Principles within controlled, GCP-aligned clinical workflows.
At Just in Time GCP, our AI framework is explicitly built on the FDA AI/ML Credibility Framework (January 2025) and reinforced by our AI Accountability Charter. Together, they guide a risk-based, human-centered approach to designing, testing, deploying, and governing AI tools used to support clinical documentation and compliance activities. These practices are also aligned with ICH E6(R3) expectations for risk-based quality management and document oversight. Below, we map our practices directly to the FDA’s 10 Guiding Principles, using the same criteria we encourage clinical trial sponsors to apply when evaluating vendors.
As a concrete example, this article references our AI-enabled Site Document Review Service, which supports TMF completeness and compliance reviews by analyzing site-level clinical trial documentation. The service uses AI to extract key information from documents related to site personnel (such as Delegation of Authority logs, FDA Forms 1572, medical licenses, CVs, financial disclosures, and training records), identify potential issues, and present results in structured reports for expert review. We designed the service to augment, not replace, qualified human judgement, and teams use it within controlled, GCP-aligned review workflows.
FDA & EMA AI Guiding Principle
Operational Application in Clinical AI Workflows
1. Human-Centric Design
AI outputs are advisory and reviewed by qualified professionals. We embed human-in-the-loop oversight, beginning with full review and evolving toward risk-based confirmation. The tool does not automate decisions or replace expert judgment.
2. Risk-Based Approach
Model risk is assessed within a defined context of use. Credibility testing, access controls, environment isolation, and mandatory human review mitigate risks related to accuracy, confidentiality, and misuse.
3. Adherence to Standards
Development follows a controlled Software Development Lifecycle (SDLC) including documented requirements, risk assessment, testing, and release approval. Infrastructure includes identity-based access control, encryption, and immutable audit logging.
4. Clear Context of Use
The AI tool supports internal TMF completeness and compliance review. It does not support clinical decision-making, patient safety determinations, or regulatory submissions. Context-of-use documentation informs risk and oversight requirements.
5. Multidisciplinary Expertise
Governance is maintained by cross-functional teams combining GCP/TMF domain expertise, AI specialists, software engineers, infrastructure security professionals, and product leadership.
6. Data Governance and Documentation
Project-specific Data Stewardship Plans define transfer, storage, processing, restriction, and deletion of client data. We process data in isolated environments, do not use it for model training, and securely destroy it at project closure.
7. Model Design and Development Practices
Structured prompt design, fixed model versions, documented processing steps, and controlled deployments are used. Teams track, review, and assess changes for impact before release.
8. Risk-Based Performance Assessment
Credibility assessments aligned with the FDA Draft AI/ML Credibility Framework evaluate whether the tool reliably identifies issues relevant to its defined context of use. False positives and missed issues are monitored and mitigated.
9. Lifecycle Management
Lifecycle controls include periodic credibility reassessment, controlled tuning, post-release monitoring, and defined decommissioning processes for client environments.
10. Clear, Essential Information
End users receive plain-language documentation describing context of use, capabilities, limitations, issue definitions, and appropriate interpretation of outputs. Known limitations and areas requiring human judgment are explicitly described.
1. Human-centric by design
First, the Site Document Review Service is intentionally designed as a human-centric, decision-support system. Its outputs are advisory and are reviewed and confirmed by qualified professionals before being used in client deliverables. Human-in-the-loop oversight is embedded throughout use. It begins with full human review and evolves toward targeted, risk-based confirmation as experience is gained. The tool does not automate decisions or replace expert judgment, reinforcing accountability and professional responsibility.
2. Risk-based approach
Second, risk management is foundational to how the tool is designed, assessed, and deployed. Model risk is evaluated in relation to the defined context of use, focusing on what could go wrong, how severe the impact would be, and how those risks are mitigated. Because risks related to accuracy, data protection, confidentiality, and misuse can affect compliance outcomes, they are addressed through credibility testing, access controls, environment isolation, and mandatory human review, consistent with a low-to-moderate risk, decision-support use case.
3. Adherence to standards
In addition, development and deployment of the Site Document Review Service adhere to relevant GCP, regulatory, technical, and cybersecurity standards applicable to software that supports clinical trial documentation and compliance activities. Controlled SDLC and deployment practices enforce requirements management, risk assessment, testing, credibility evaluation, and release approval, while the system architecture incorporates identity-based access control, encryption in transit and at rest, environment isolation, and immutable audit logging. Together, these controls operationalize the FDA Draft AI/ML Credibility Framework and align with the AI Accountability Charter’s commitments to ethical, transparent, and accountable AI use.
4. Clear context of use
For example, the context of use for the Site Document Review Service is narrowly defined and consistently documented. The tool supports internal TMF completeness and compliance reviews by identifying missing, expired, misclassified, or anomalous site personnel documentation. It does not support clinical decision-making. It does not support patient safety determinations or regulatory submissions. This clarity informs model risk assessment, credibility testing, and the requirement for ongoing human oversight.
5. Multidisciplinary expertise
Similarly, the tool is developed and governed by a cross-functional team that brings together deep GCP and TMF domain expertise, built on many years of hands-on experience, alongside AI specialists, software engineers, infrastructure and security professionals, and product management leadership. This combination ensures that clinical, regulatory, technical, and usability considerations are addressed together rather than in isolation, and that AI capabilities are applied in a way that reflects real-world clinical documentation practices.
6. Data governance and documentation
Data governance is addressed through formal, project-specific Data Stewardship Plans. These plans define how client data is transferred, stored, processed, restricted, and deleted. Client data is processed only within secured, isolated Azure environments, is not used to train AI models, and is deleted at project closure with issuance of a Certificate of Destruction. These controls align with FDA expectations for data integrity and confidentiality and are reinforced by the AI Accountability Charter’s commitments to data protection and minimization.
7. Model design and development practices
The model used by the Site Document Review Service is constrained through structured prompt design, fixed model versions, documented processing steps, and controlled deployments. In addition, development follows a defined Software Development Lifecycle that requires documented requirements, architecture design, risk assessment, testing, and release approval. Changes to prompts, infrastructure, or model components are tracked, reviewed, and assessed for impact rather than introduced ad hoc.
8. Risk-based performance assessment
Performance is evaluated using a credibility assessment framework aligned with the FDA Draft AI/ML Credibility Framework. As a result, testing focuses on whether the tool reliably identifies documentation issues that matter for its defined context of use, using controlled datasets, automated database comparisons, and structured review of discrepancies. Performance risks such as false positives, missed issues, and misidentification are explicitly identified and mitigated through credibility testing, fixed model versions, and mandatory human review.
9. Lifecycle management
Finally, lifecycle management for the Site Document Review Service is formally defined and governed through an established Software Development Lifecycle SOP. It extends beyond initial development to include periodic credibility reassessment, controlled tuning, post-release monitoring, and secure environment teardown. Credibility assessments are re-executed following changes to the tool or deployment environment, and client environments are intentionally temporary with defined decommissioning processes.
10. Clear, essential information
In addition, clear, plain-language information is provided to end users to explain the tool’s context of use, capabilities, limitations, and appropriate interpretation of outputs. End-user training materials define results using precisely specified issue types, document status rules, and severity weights, ensuring outputs are consistent and unambiguous. Known limitations and areas requiring human judgment are explicitly described, supporting informed and appropriate use of the tool’s outputs.
Closing perspective
The FDA/EMA’s Guiding Principles of Good AI Practice provide a practical lens for evaluating both emerging AI technologies and the organizations that build them. By defining context of use, assessing risk appropriately, applying credibility-based evaluation, protecting data, and maintaining lifecycle accountability, it is possible to adopt AI responsibly even as the technology continues to evolve. The same questions clinical trial sponsors should ask their vendors are the questions responsible vendors should already be prepared to answer.
By Kathie Clark, Industry Expert; Technology & Innovation Partner, Just in Time GCP
Kathie is an integral part of Just in Time GCP’s growing innovation team. As Founder & CEO, Donna Dorozinsky has said “Kathie offers a strategic vision for innovative technology application that embraces AI and clinical data analytics, allowing Just in Time GCP to translate increased quality and efficiencies to our partners.” Click here to learn more about how Kathie and Just in Time GCP are applying AI with purpose, not hype, and realizing results.
