Summary

On December 6, 2018, Commissioner Scott Gottlieb announced the FDA’s highly anticipated new strategic framework for assessing the potential use of real-world evidence (RWE) in connection with the agency’s drug and biologic review program. The Framework follows a series of guidance documents, demonstration projects and other initiatives that FDA has already undertaken to explore and facilitate the use of real-world data (RWD) and RWE in the context of post-market surveillance activities, clinical trials and medical device regulatory decision making. 

The Framework outlines key action items for FDA’s further assessment of the use of RWE and RWD and development of applicable standards that will be of interest to manufacturers, RWD companies and health care providers interested in harnessing RWD and RWE to enhance patient safety and treatment, and other stakeholders.

In Depth

Background

As FDA Commissioner Scott Gottlieb noted, RWD and RWE is a “top strategic priority” for the agency. FDA issued the Framework in response to the 21st Century Cures Act (Cures Act), which added section 505F (21 USC § 355g) to the Federal Food, Drug, and Cosmetic Act (FDCA), requiring FDA to establish a program to evaluate the potential use of evidence from clinical experience to help (1) support the approval of a new indication for a drug already approved under section 505(c) (21 USC § 355(c)) of the FDCA or (2) support or satisfy drug post-approval study requirements. FDA announced it will also apply the Framework to biological products licensed under the Public Health Service Act. 

The Framework is not FDA’s first foray into the issue of RWD and RWE. Key prior initiatives and guidance documents include:

• The use of the Sentinel System (as discussed below); 

• The Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices guidance (RWE for Medical Devices guidance), issued on August 31, 2017; 

• The Best Practices for Conducting and Reporting Pharmacoepidemiologic Safety Studies Using Electronic Healthcare Data guidance, issued May 2013, outlining FDA’s perspective on the use of RWD available in electronic health care data systems for safety studies; and

• The agency’s MyStudies mobile application, released in November 2018 (as discussed below). 

The Framework, however, is meaningful in its establishment of a more definitive roadmap for FDA’s development of standards for the use of RWD and RWE. In particular, the Framework outlines various areas in which FDA will evaluate the use of RWD and RWE; FDA’s plan for evaluating how RWD can be incorporated into study designs and the regulatory considerations for such use of RWD; and FDA’s plan for developing guidance in a number of key areas, including on data standards and additional sources of RWD to help address current gaps in the capture of key information.  

FDA plans to use the Framework to evaluate both the potential use of RWE to support labeling changes, such as adding or modifying an indication, and to satisfy post-marketing requirements. FDA stated it intends to continue to engage with internal and external stakeholders to adapt or develop standards and implementation strategies, including through future public meetings and other forums.

Overview of Framework

Key Definitions

The Framework identifies RWD and RWE as follows:

• Real World Data (RWD): data relating to patient health status or delivery of health care routinely collected from a variety of sources, such as registries, collections of electronic health records (EHRs), and administrative and medical claims databases. 

• Real World Evidence (RWE): clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of RWD through, e.g., randomized trials (large simple trials, pragmatic clinical trials) and observational studies (retrospective or prospective).

Evidence That Will Be Considered RWE

FDA distinguishes RWE from traditional clinical data. Evidence from traditional clinical trials will not be considered RWE. The agency generally considers traditional clinical trials to be studies supported by a research infrastructure largely separated from routine clinical practice, designed to control variability (e.g., through restrictive eligibility criteria) and to maximize data quality (e.g., through randomization and double-blinding). 

FDA notes, however, that RWD could arise from hybrid design trials (e.g., where certain trial data is collected from RWD sources, such as EHRs or claims data, while other trial data is collected from study-specific interventions) or pragmatic design trials (i.e., trials that include elements that more closely resemble routine clinical practice).

• Key FDA Action Item: As part of the Framework, FDA will evaluate the strengths and limitations of pragmatic approaches to study design that use RWD, including the: (1) types of interventions and therapeutic areas that may be well-suited to routine clinical care settings, (2) quality of data that can be captured in these settings, (3) number of patients that can be accessed (particularly when outcomes are rare) and (4) variations inherent in clinical practice.  

Likewise, FDA posits that non-interventional observational studies (either retrospective or prospective) could generate RWE. According to the agency, observational studies typically share certain characteristics, such as the use of outcomes when there is less diagnostic variability and where the lack of blinding is less likely to influence outcomes (e.g., myocardial infarction or stroke). Where blinding is infeasible, FDA will seek to identify situations when researchers can potentially control bias by using outcomes that are less likely to be influenced by knowledge of treatment assignment, such as clinically objective outcomes (e.g., stroke or tumor size). 

FDA notes that because observational studies rely on treatment assignment by a physician rather than at random, they pose a challenge for establishing causal inference, and recent efforts to replicate randomized trial results with observational studies have shown divergence in results. 

• Key Action Item: FDA intends to focus on a number of areas that could affect the ability to draw a reliable causal inference, including: (1) characteristics data improve the chance of a valid result; (2) characteristics of study design and analysis that improve the chance of a valid result (e.g., whether an active comparator can improve the chance of a valid result and whether there is a role for non-inferiority designs); and (3) the sensitivity analyses and statistical diagnostics that should be pre-specified to generate RWE for effectiveness. 

Use of RWD and RWE to Support Effectiveness Determinations 

FDA notes it has a long history of using RWE to monitor and evaluate the post-market safety of drug products. The agency’s own primary source for pharmacoepidemiologic queries and studies (i.e., examining the utilization and effects of drugs in large numbers of people) has been medical claims and pharmacy dispensing data collected through the Sentinel System. The agency’s use of RWE to support effectiveness determinations, however, has been much more limited. To date, FDA has accepted RWE to support drug product approvals in limited instances that are primarily in oncology and rare diseases. 

• Key Action Item: To facilitate the use of RWD and RWE in decisions regarding product effectiveness, FDA announced a three-part approach to evaluating whether RWD can generate RWE to support regulatory decisions regarding the effectiveness of a drug or biologic product. Specifically, FDA will consider:

1. 1. Whether the RWD are fit for use;

   2. Whether the trial or study design used to generate RWE can provide adequate scientific evidence to answer or help answer the regulatory question; and

   3. Whether the study conduct meets FDA regulatory requirements (e.g., for study monitoring and data collection).

FDA intends to build out this approach through stakeholder engagement, the use of internal processes that engage senior agency leadership, demonstration projects, and additional guidance documents to assist sponsors interested in using RWE in support of their work.

Applying similar criteria as in its RWE for Medical Devices guidance, FDA stated that the strength of RWE depends on the reliability and relevance of the underlying data. The reliability assessment considers the validity and integrity of the data (i.e., data assurance and data quality control), whereas the relevance assessment considers whether the data are fit for the purpose and whether the data capture is relevant. Under the Framework, FDA also intends to evaluate whether its existing guidance documents related to the use of electronic data in clinical investigations are relevant to different study designs using RWD to generate RWD for drug product effectiveness. 

Fitness of Data and Gaps in RWD Sources

A key component of FDA’s ability to use RWD and RWE in regulatory decision making is ensuring that the RWD and RWE is suitable for the proposed use. As noted above, FDA’s RWE expertise has, to date, been developed largely from working with claims—rather than EHR—data. 

With respect to claims data, FDA notes that it has a good understanding of the strengths and limitations of such data. For EHR data, which tends to include much more robust medical information, significant challenges arise from the fact that key information is often captured as unstructured data (e.g., in physician notes) that cannot be readily harnessed for use—an issue that is exacerbated insofar as vast amounts of EHR data is necessary for a particular purpose. FDA’s recognition of the challenges that EHR data presents is evident in guidance documents it has previously issued, including its Use of Electronic Health Record Data in Clinical Investigations guidance. Additionally, FDA notes that EHR data often fails to uniformly code or include laboratory test results and may not fully capture patient outcomes and the progression of symptoms and conditions.

Another limitation of claims and EHR data is the inability to effectively capture data regarding patient experience. Thus, the use of registries and EHR systems with patient-targeted data collection platforms, and digital health tools including wearables and mobile medial apps, will be important mechanisms for the agency to consider in filling current gaps in critical RWD. Indeed, the MyStudies application, developed to support informed consent as well as secure data collection from patients at multiple study sites or data partners, represents an effort by the agency to facilitate investigators’ ability to collect real-time patient experience data to support FDA-regulated studies. 

• Key Action Items: FDA indicated its plans to undertake the following to address issues relating to the suitability of RWD and RWE and gaps in RWD sources:

  • Issue guidance on how to assess the reliability and relevance of RWD from medical claims and EHRs used to generate RWE regarding drug product effectiveness; 

  • Examine how to assess the reliability and relevance of registry data and international electronic data; 

  • Review and, where applicable, publish guidance on potential gaps in RWD sources and strategies to address them (e.g., mobile technologies, electronic patient-reported outcome tools, wearables, biosensors). 

FDA’s Commitments under the Framework

FDA noted it is actively developing RWD data standards (rules regarding how a particular type of data should be defined, formatted or exchanged between computer systems) for regulatory use.

FDA also outlined a number of commitments and plans for the Framework related to study designs using RWD to support effectiveness and related regulatory considerations. In particular, FDA indicated it intends to: 

• Evaluate the potential role of observational studies in contributing to evidence of drug product effectiveness;

• Develop guidance on considerations for designing clinical trials that include pragmatic design elements and that generate evidence of effectiveness for regulatory decisions. FDA will explore pragmatic approaches to each stage of a clinical trial, including recruiting and enrollment of patients, strategies for facilitating interventions and approaches to assessing outcomes;

• Consider guidance on the use of RWD to generate external control (e.g., historical control) arms;

• Issue guidance about observational study designs using RWD, including whether and how these studies might provide RWE to support product effectiveness in regulatory decision making. FDA will also consider reporting requirements for such studies, analogous to ClinicalTrials.gov reporting requirements for clinical trials, used to support effectiveness determinations in order to promote transparency;

• Consider whether existing guidance documents that address the use of electronic data in clinical investigations adequately address concerns relevant to different study designs using RWD to generate RWE or whether additional guidance is needed;

• Examine how FDA’s regulatory requirements are applied to data from randomized clinical trials that are integrated into the health care system and observational studies when they are intended to generate RWE for regulatory decision making; 

• Consider whether its current guidance adequately addresses inspection-related concerns relevant to different sources of RWD used to generate RWE regarding the safety and effectiveness of drug products, e.g., consent and privacy; and

• Finalize the Use of Electronic Records and Electronic Signatures in Clinical Investigations Under 21 CFR part 11 - Questions and Answers guidance and consider its applicability to different study designs. FDA will also issue additional guidance as needed on regulatory considerations raised by different study designs using RWD to generate RWE.

Key Takeaways

The Framework reveals a number of priorities for FDA as it continues to evaluate the proper role and use of RWD and RWE. While the Framework is expressly limited to the drug and biologic review programs, the considerations described in the Framework may be reflective of areas of focus for the agency’s regulatory strategy more broadly, including with respect to medical device regulation. In that regard, stakeholders may wish to consider the following areas of focus by FDA:

• The development and use of digital health tools and other platforms to enhance the collection of patient-reported outcomes;

• Solving for challenges relating to EHR data, including the ability to efficiently and effectively abstract unstructured data and normalize large amounts of EHR data collected from different systems and multiple entities; and

• Properly documenting the implementation of FDA standards that will be developed under the Framework to demonstrate compliance during FDA’s review of submitted RWD or RWE, including through policies and procedures and robust documentation of data standards, data control and other data management mechanisms.

The ability to leverage RWD and RWE for discrete components of clinical trials and other regulatory uses (such as hypothesis generation, trial feasibility assessments, and as part of hybrid clinical trials). The role of RWE to support marketing claims is particularly relevant in light of the Medical Product Communications That Are Consistent With the FDA-Required Labeling – Questions and Answers guidance. While the Framework and proposed future action items address a number of issues, the Framework leaves open questions regarding the use of RWE to support marketing claims.