Real World Evidence: A Potential Solution

By Christopher T. Fulmer 

Introduction

The average cost of developing a drug or biologic that obtains market approval is approximately $2 billion with an additional $312 million expended in postmarket monitoring,[1] and the average time to develop, and obtain FDA approval for, a drug or biologic is approximately ten years.[2] While the amount spent on the research and development of a drug or biologic continues to increase, the overall efficiency of research and development has drastically declined. The number of new drugs or biologics approved per billion U.S. dollars spent on research and development has halved roughly every nine years since 1950, falling around eighty-fold in inflation-adjusted terms,[3] and only approximately ten percent of drugs and biologics are actually approved.[4]

The staggering and escalating costs of developing a drug or biologic coupled with the relatively low success rate of obtaining FDA approval has caused pharmaceutical and biotechnology companies to search for ways to decrease research and development costs of drugs and biologics while increasing efficiency. Advances in technology are providing pharmaceutical and biotechnology companies with a potential new resource to cultivate and integrate into various aspects of the development of drugs and biologics. This resource is called healthcare data and often referred to as real-world data (RWD) and real-world evidence (RWE). The McKinsey Global Institute estimates that utilizing real-world data and real-world evidence to make better informed decisions “could generate up to $100 billion in value annually across the US health-care system, by optimizing innovation, improving the efficiency of research and clinical trials, and building new tools for physicians, consumers, insurers, and regulators to meet the promise of more individualized approaches.”[5]

Consequently, pharmaceutical and biotechnology companies, along with the industry as a whole, are increasingly turning towards utilizing healthcare data in an effort to harness its potential value. The regulation of the usage of this data and the potential benefits to be derived are in their infancy and evolving rapidly, but FDA’s willingness to explore a totality of the evidence approach could revolutionize the development of drugs and biologics as well as the approval process.

Increasing Use of Healthcare Data

Pharmaceutical and biotechnology companies have started creating large databases to leverage the knowledge that can be gathered from various healthcare records and data to improve the design and efficiency of clinical trials in the following ways:

  1. Generating hypotheses for testing in randomized controlled trials;
  2. Identifying drug development tools, such as biomarker identification;
  3. Assessing trial feasibility by examining the impact of planned inclusion and exclusion criteria within a geographical area or at a particular site;
  4. Assembling geographically distributed research cohorts specifically in drug development for rare diseases or targeted therapeutics;
  5. Identifying prognostic indicators or patient baseline characteristics for enrichment or stratification; and
  6. Identification of underserved markets and treatment inefficiencies.[6][7]

The potential financial significance alone this data can have on improving the design and efficiency of clinical trials is astonishing. For example, in 2017, the National Institutes of Health estimated that more than eighty percent of clinical trials in the U.S. fail to meet their enrollment timelines,[8] and in 2015, approximately $1,917,550,000 was wasted on non-performing sites.[9] Harnessing the potential value of the data could save millions of dollars in just this one area.

Regulatory and Data Issues

While healthcare data is increasingly being leveraged to improve the design and efficiency of clinical trials, the use of such data to support the evaluation of the effectiveness of drugs and biologics raises various regulatory issues. Section 505(d) of the Food, Drug, and Cosmetic Act specifies a drug’s effectiveness must be shown by substantial evidence defined as “evidence consisting of adequate and well-controlled investigations, including clinical investigations, by experts qualified by scientific training[,]”[10] and 21 C.F.R. § 314.126 sets forth the characteristics of an adequate and well-controlled study, specifying five types of generally recognized controls—placebo concurrent, dose-comparison concurrent, no treatment concurrent, active treatment concurrent, and historical.[11] Furthermore, under Section 351 of the Public Health Service Act, licenses for biologics are issued only upon a showing that the products are safe, pure, and potent.[12] Potency has long been interpreted to include effectiveness, and FDA has generally considered “substantial evidence” of effectiveness to be necessary to support licensure of a biologic.[13] FDA, in general, has interpreted these provisions to require two adequate and well-controlled clinical trials to demonstrate substantial evidence, and randomized clinical trials have become known as the gold standard.[14]

However, with the passage of the Food and Drug Administration Modernization Act of 1997 and FDA’s 1998 Guidance titled, “Providing Clinical Evidence of Effectiveness for Human Drug and Biological Products,” a degree of flexibility was added to the substantial evidence requirement.[15] Both the Act and FDA’s guidance state the substantial evidence requirement for effectiveness could be met by a single clinical trial plus confirmatory evidence,[16] thereby enabling the use of real-world data and real-world evidence.

Historically, FDA has limited the use of real-world data and real-world evidence to historical controls and for postmarket surveillance as 21 C.F.R. § 314.126(v) and the Food and Drug Administration Amendments Act of 2007 provide the bases to use such data for these two purposes.

Section 314.126(v) specifically defines one type of adequate and well-controlled study to be one consisting of a historical control where “the results of treatment with the test drug are compared with experience historically derived from the adequately documented natural history of the disease or condition, or from the results of active treatment, in comparable patients or populations.”[17] Historical controls have been successfully used where the disease or condition is rare or life-threatening, such as various cancers. For example, with Pfizer’s Bavencio (avelumab), a treatment for Merkel cell carcinoma, efficacy was substantiated by matched historical controls from a retrospective electronic health record review supported by literature reviews.[18]

With postmarket surveillance, the Food and Drug Administration Amendments Act of 2007 required FDA to develop a system for postmarket risk identification and analysis of drugs resulting in the development of the Sentinel System, an electronic healthcare database used to assess the safety of approved medical products.[19] As of August 2018, the Sentinel System contained data on more than 100 million patients and 310.8 million unique patient identifiers within a network of eighteen data partners and collaborating institutions.[20] FDA uses the system before the approval of a drug or biologic to provide safety information and designs studies to examine safety questions after approval.[21] FDA, for example, has “used the Sentinel System to evaluate the risk of [a] stroke after using antipsychotics . . . [and] the risk of seizures after using ranolazine. . . .”[22]

Even though the use of real-world data and real-world evidence has been successful in historical control arms and with postmarket surveillance, FDA has limited its use because of the regulatory requirements of substantial evidence and adequate and well-controlled investigations along with the challenges associated with real-world data, such as:

  1. Quality of Data: Much of real-world data, such as administrative and claims data and electronic healthcare data, is not collected and organized with the goal of supporting research. Furthermore, policies and standards promoting and facilitating the input of high quality real-world data and the collection of real-word data vary widely throughout the world, and even within specific regions, resulting in data fragmentation affecting the quality and accuracy of data.[23]
  2. Missing Data: There is no standardized manner of managing missing data, which potentially results in utilizing inaccurate raw data or not utilizing data that can be adjusted by implementing certain accepted scientific measures, such as median imputation or multivariate imputation by chained equations.[24]
  3. Collection Issues: The collection of data is timely and costly as the various sources are inadequately connected and often incompatible.[25]
  4. Bias Selection: As with clinical research data, there are concerns regarding the bias selection of real-world data where a company or investigator participates in data torturing by deciding on a hypothesis and manipulating the data to fit the hypothesis.[26]
  5. Privacy Issues: When abstracting data from patient platforms and social media sites, physicians are often concerned about privacy issues. Patients want the ability to consent to the use of their information[27] and will only share data if they are confident their information will remain private and secure.
  6. Education Issues: Many people in product development have limited or no experience conducting observational and non-interventional research.[28] Companies and even regulatory entities need to train personnel regarding the utilization and collection of real-world data and real-world evidence to enhance the value of such information.[29]Additionally, all stakeholders, in general, need to be educated that the potential benefits of real-world data collection and analysis outweigh the potential risks.
  7. Analytical Issues: The lack of standardization regarding real-world data and real-world evidence analytics can potentially lead to poor quality analysis, limited transparency into methods, inability to integrate data from disparate sources, and biased results.[30]
  8. Gold Standard Bias: For decades, multiple randomized clinical trials have been the predominate manner to demonstrate a drug or biologic’s efficacy and safety, resulting in the consensus that even though real-word data and real-world evidence are proving to be useful tools, randomized clinical trials are still the gold standard to demonstrate efficacy and safety.[31]

21st Century Cures Act and FDA’s Framework

With the intent to accelerate medical product development and bring new innovations and advances faster and more efficiently to patients, Congress enacted the 21st Century Cures Act on December 13, 2016. Section 3022 of the 21st Century Cures Act “amends the FD&C Act to establish a program at [the] FDA to evaluate the potential use of real-world evidence to (a) help support the approval of new indications for an approved drug and (b) help support or satisfy post approval (i.e., Phase 4) study requirements. The section [also] defines real world evidence to mean ‘data regarding the usage, or the potential benefits or risks, of a drug derived from sources other than clinical trials. . . .’”[32]

Since the passage of the 21st Century Cures Act, FDA issued a Framework regarding a Real-World Evidence Program (hereinafter referred to as “Framework”) setting forth: (1) what is real-world data and real-world evidence; (2) a test on how to evaluate real-world data and real-world evidence; and (3) the scope of the program.

“Real world data is data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources.”[33] Real-world data can be broken down into four categories to provide a more in-depth understanding of how much potential data is available, the varying sources of the data, and the varying quality of the data. First, clinical data is patient-level data pulled from electronic medical records and patient registries reflecting how a patient is treated in the real world, including lab values, diagnoses, notes, and other information from visits to physicians and other providers.[34] Second, administrative and claims data is patient-level data collected for non-clinical purposes, such as information commonly used for billing purposes by providers to insurers and other payors which can include diagnoses, services provided, costs, and other data as well as data collected for tracking purposes, such as disease registries and patient surveys.[35] Third, patient-reported data is collected and reported by a patient describing the patient’s experience and is often reported through mobile and wearable patient devices.[36] The fourth category is non-traditional health-related digital sources, such as social media posts or sites, which are providing newer and ever-increasing sources of patient-level health data.[37]

“Real world evidence is the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD.”[38] Real-world evidence can be derived from hybrid design trials, pragmatic trials, and observation studies. A hybrid design trial is a study design with a “dual focus on assessing clinical effectiveness and implementation. . . . [There are] three types of hybrid [design] trials: (1) testing effects of a clinical intervention on relevant outcomes while observing and gathering information on implementation; (2) dual testing of clinical and implementation interventions/strategies; and (3) testing of an implementation strategy while observing and gathering information on the clinical intervention’s impact on relevant outcomes.”[39] Pragmatic trials are clinical trials that measure effectiveness by testing a full range of patients who might be treated with the drug or intervention (the degree of the beneficial effect of a drug or intervention in real clinical practice).[40] “Observational studies are non-interventional clinical study designs,” meaning researchers do not assign patients to a treatment but observe patients and measure certain results to determine health outcomes.[41] There are two types of observational studies—retrospective and prospective. A retrospective observational study identifies the population and determines the exposure or treatment from historical data,[42] and in a prospective observational study, “the population of interest is identified at the start of the study and the exposure [or] treatment and outcome data are collected from that point forward.”[43]

When evaluating real-world data and real-world evidence for regulatory decisions, FDA has designed a three-prong test. First, FDA determines the real-world data’s fitness by examining the data’s reliability, which must be “collected and maintained in a way that provides an appropriate level of reliability,”[44] and by examining the data’s relevance, meaning the data should be “suitable to address the specific regulatory question of interest.”[45] While the healthcare industry and FDA are working towards more standardization and compatibility of real-world data and data analytics in an effort to realize the true value of the data, the determination of the data’s reliability and relevance will always be a key factor in using real-world data and the corresponding real-world evidence in regulatory decisions.

Next, FDA evaluates whether “the study used to generate the RWE can provide adequate scientific evidence to answer or help answer the regulatory questions.”[46] Depending on the study’s design, slightly different factors are evaluated. FDA will consider the following four important factors when evaluating the strengths and limitations of hybrid and pragmatic study designs:

  1. “What types of interventions and therapeutic areas might be well-suited to routine clinical care settings?
  2. What is the quality of the data that can be captured in these settings?
  3. How many patients can be accessed?
  4. What are the variations inherent in clinical practice?”[47]

For observational study designs, FDA will consider the following factors when evaluating the strengths and limitations of observational study designs:

  1. “What are the characteristics of the data that improve the chance of a valid result?
  2. What are the characteristics of the study design and analysis that improve the chance of a valid result?
  3. What sensitivity analyses and statistical diagnostics should be prespecified for observational studies using RWD to generate RWE for effectiveness?”[48]

The final prong of FDA’s test is a determination of whether the approach used in a particular case meets FDA’s regulatory requirements, such as established standards for data collection, informed consent, and study monitoring.[49]

The Framework also sets forth the scope of the Real-World Evidence Program, centering on evaluating many of the areas where real-world evidence will have a significant beneficial impact on the substantial evidence requirement of demonstrating a drug or biologic’s effectiveness. FDA is evaluating the use of real-world evidence[50]:

  1. In support of the effectiveness or safety for a new product’s approval
    For example, Amgen’s Blincyto (blinatumomab), a treatment for B-cell precursor acute lymphoblastic leukemia, obtained accelerated approval as the drug’s efficacy was substantiated by matched historical controls from patient data extracted from over 2,000 patient records from European Union and United States treatment sites.[51]
  2. To support changes to labeling about drug product effectiveness, including: 
    1. Adding or modifying an indication, such as a change in dosage, dosage regimen, or route of administration
      FDA’s approval expanding the indication of NovoSeven® RT, a product used to treat hemophilia, for the treatment of Glanzmann’s thrombasthenia (GT) with refractoriness was based upon evidence collected from the global Glanzmann’s Thrombasthenia Registry and the Hemostasis & Thrombosis Research Society Registry as the data supported the effectiveness and safety of NovoSeven®RT in the treatment of bleeding, and for the prevention of bleeding, during surgical procedures in patients with GT.[52]
    2. Adding a population
      On April 4, 2019, FDA approved the use of Pfizer’s Ibrance in men with breast cancer based largely on Pfizer’s analysis of real-world data from electronic health records and postmarketing reports demonstrating the efficacy and safety profile in men was consistent with women treated with Ibrance.[53]
    3. Adding comparative effectiveness or safety information
      For example, when RotaShield, a rotavirus vaccine, was withdrawn from the market due to an increased risk of intussusception, concern rose regarding RotaTeq, another rotavirus vaccine. Based on a review of the Sentinel System’s data, FDA identified the administration of RotaTeq caused an increased risk of intussusception, which was not detected during clinical trials prior to approval. This information led to FDA requiring an applicable warning to be included in RotaTeq’s label.[54]
  3. In postmarket monitoring and evaluating a drug or biologic’s safety
    Increasingly, FDA has approved drugs and biologics conditioned upon the performance of postmarket studies. When FDA approved Tagrisso (osimertinib), a treatment for metastatic epidermal growth factor receptor (EGFR) T790M mutation-positive-non-small-cell lung cancer, the agency required AstraZeneca Pharmaceuticals, LP—the manufacturer—to provide the overall response rate data “from one or more ‘realworld’ cohorts of a minimum of 100 patients . . . selected for treatment on the basis of an EGFR T790M mutation positive result. . . .”[55]

FDA’s Framework and three-prong test for evaluating real-world data and real-world evidence reflects FDA’s willingness to use the flexibility afforded under the Food and Drug Administration Modernization Act of 1997 to evaluate the totality of the evidence afforded by clinical trials, real-word data, and real-world evidence complementing one another to provide the substantial evidence necessary to demonstrate a drug or biologic’s effectiveness. Furthermore, FDA’s Framework acknowledges the importance of data standards and implementation strategies and will seek to strengthen these areas with input from industry stakeholders to improve the reliability and relevance of real-world data and real-world evidence in future regulatory decisions.

Conclusion

Pharmaceutical and biotechnology companies are already capturing some of real-world data’s potential and value to improve the efficiency of trial designs and site selections. FDA has also seen the successful impact real-world evidence can have on regulatory decisions through the use of historical controls and postmarket surveillance. However, the potential of real-world data and real-world evidence is far greater.

The reliability and relevance of both real-world data and real-world evidence is the biggest obstacle faced by pharmaceutical companies, biotechnology companies, and FDA in harnessing the true untapped value real-world data and real-world evidence could have on the development of drugs and biologics. While the quality of real-world data will always need to be evaluated to determine its fitness for use, with the modernization and standardization of healthcare databases and data analytics, real-world evidence and real-world data will become more reliable and will be an increasing part of designing clinical trials, site location and recruitment, forming part of the substantial evidence needed to obtain approval, and postmarket surveillance.

Traditional clinical trials, real-world evidence, and real-world data all have shortcomings, so FDA remaining flexible and using a totality of the evidence approach, where knowledge gained from each source complements the knowledge gained from the other sources, is imperative. The totality of the evidence approach should result in the research and development of drugs and biologics becoming more efficient, thereby reducing costs and increasing the discovery of new or more effective drugs and biologics used to treat underserved segments of the population and to treat previously untreatable diseases or illnesses.

 

REFERENCES

[1] “Using Real World Evidence to Accelerate Safe and Effective Cures,” Bipartisan Policy Center, June 2016, P 11, https://bipartisanpolicy.org/wp-content/uploads/2019/03/BPC-Health-Innovation-Safe-Effective-Cures.pdf.

[2] U.S. Department of Veteran Affairs, “How Long Does The FDA Take to Approve a Drug,” last accessed June 16, 2020, https://www.hiv.va.gov/patient/clinical-trials/drug-approval-process.asp.

[3] Jack W. Scannell, Alex Blanckley, Helen Boldon, et al., “Diagnosing the Decline in Pharmaceutical R&D Efficiency”, Nature Review/Drug Discovery, March 2012, P 192-193, https://www-nature-com.ezproxy.neu.edu/articles/nrd3681.pdf.

[4] Scannell, “Diagnosing the Decline,” 193.

[5] Jamie Cattell Sastry, Chilukuri, and Michael Levy, “How Big Data Can Revolutionize Pharmaceutical R&D,” McKinsey & Company, April 1, 2013, last accessed June 20, 2020, https://www.mckinsey.com/industries/pharmaceuticals-and-medical-products/our-insights/how-big-data-can-revolutionize-pharmaceutical-r-and-d#.

[6] Rayhnuma Ahmed, Emelly Rushi, Aaron Stern, et al., “Using Real-World Evidence to Optimize Clinical Trials,” SHYFT ANALYTICS, last accessed June 20, 2020, https://www.shyftanalytics.com/wp-content/uploads/2018/01/SHYFT-Analytics_Using-Real-World-Evidence-to-Optimize-Clinical-Trials_2018.pdf.

[7] Framework for FDA’s Real-World Evidence Program, U.S. Food & Drug Administration, 7, https://www.fda.gov/media/120060/download.

[8] Hamza Khan, “Innovation in Clinical Trials – What’s the Future of Patient Recruitment? ,” ClinEdge, April 20, 2017, https://news.clin-edge.com/innovation-clinical-trials-whats-future-patient-recruitment.

[9] Ahmed, “Using Real-World Evidence.”

[10] The FD&C Act Section 505(d); 21 U.S.C. § 355(d).

[11] 21 CFR 314.126.

[12] “Demonstrating Substantial Evidence of Effectiveness for Human Drug and Biological Products Guidance for Industry,” FDA Draft Guidance, U.S. Food and Drug Administration, December 2019, 3 https://www.fda.gov/media/133660/download.

[13] U.S. Food and Drug Administration, “Demonstrating Substantial Evidence,” 3.

[14] U.S. Food and Drug Administration, “Demonstrating Substantial Evidence,” 4.

[15] “Demonstrating Substantial Evidence of Effectiveness for Human Drug and Biological Products,” U.S. Food and Drug Administration, last accessed June 16, 2020, https://www.fda.gov/regulatory-information/search-fda-guidance-documents/demonstrating-substantial-evidence-effectiveness-human-drug-and-biological-products.

[16] U.S. Food and Drug Administration, “Demonstrating Substantial Evidence of Effectiveness for Human Drug and Biological Products.”

[17] 21 CFR 31.126(v).

[18] Bridget Silverman, Ben Gutierrez, Patrice Verlillat, “How Real-World Evidence Is Playing Out In The Real World,” Pharm Intelligence, Pink Sheet, Slide 11, https://pharmaintelligence.informa.com/resources/product-content/sitecore/shell//~/media/informa-shop-window/pharma/2019/files/pdf/how-rwe-is-playing-out-in-the-real-world-webinar-slides.pdf.

[19] “FDA’s Sentinel Initiative- Background,” U.S. Food and Drug Administration, last accessed June 17, 2020, https://www.fda.gov/safety/fdas-sentinel-initiative/fdas-sentinel-initiative-background.

[20] Snapshot of Database Statistics, Sentinel, last accessed June 17, 2020, https://www.sentinelinitiative.org/sentinel/data/snapshot-database-statistics.

[21] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 7.

[22] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 7.

[23] Olivia Cavlan, Sastry Chilukuri, Matthias Evers, et al., “Real-World Evidence: From Activity to Impact,” McKinsey & Company, May 7, 2018, last accessed on June 20, 2020, https://www.mckinsey.com/industries/pharmaceuticals-and-medical-products/our-insights/real-world-evidence-from-activity-to-impact-in-healthcare-decision-making.

[24] Shashank Shekhar Rai, “3 Methods to Handle Missing Data,” Oracle Al & Data Science Blog, February 28, 2019, https://blogs.oracle.com/datascience/3-methods-to-handle-missing-data.

[25] Ashraf Ali, Anto Vijay Kanth, Rituraj Mohanty, et al., “Real World Evidence: An Overview of its Importance in the Current Scenario,” Int. J. Pharm. Sci. Rev. Res., 39(2), July – August 2016, https://www.researchgate.net/publication/307566005_Real_World_Evidence_An_Overview_of_its_Importance_in_the_Current_Scenario.

[26] James L. Mills, “Data Torturing”, Ethical and Regulatory Aspects of Clinical Research Readings and Commentary”, The John Hopkins University Press, 2003, Page 397.

[27] Tom O’Connor, Berkery Noyes, “How Real-World Data and Real-World Evidence are Transforming the Healthcare Industry,” Health IT Outcomes, August 21, 2009, https://www.healthitoutcomes.com/doc/how-real-world-data-and-real-world-evidence-are-transforming-the-healthcare-industry-0001.

[28] “Barriers and Disincentives to the Use of Real-World Evidence and Real-World Data,” Examining the Impact of Real-World Evidence on Medical Product Development, National Academies Press (US), February 6, 2019, https://www.nap.edu/read/25352/chapter/5.

[29] Ali, “Real World Evidence: An Overview of its Importance.”

[30] Calvan, “Real-World Evidence” & O’Conner, “How Real-World Data.”

[31] O’Conner, “How Real-World Data.”

[32] Michelle L. Butler and James E. Valentine, “The President Signs 21st Century Cures into Law; Highlights of Drug and Biologic Related Provisions (Part One) ,” FDALAWBLOG.net, December 13, 2016, last accessed April 15, 2020, http://www.fdalawblog.net/2016/12/the-president-signs-21st-century-cures-into-law-highlights-of-drug-and-biologic-related-provisions-p/.

[33] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 4.

[34] Calvan, “Real-World Evidence.”

[35] Calvan, “Real-World Evidence.”

[36] Calvan, “Real-World Evidence.”

[37] Calvan, “Real-World Evidence.”

[38] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 4.

[39] Karen Staman, Hybrid Designs, NIH Collaboratory Living Textbook, last accessed June 20, 2020, https://rethinkingclinicaltrials.org/chapters/design/designing-implementation-dissemination-mind-top/post-5598/.

[40] The Free Dictionary by Farlex, last accessed, June 17, 2020, https://medical-dictionary.thefreedictionary.com/pragmatic+trial.

[41] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 6.

[42] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 6.

[43] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 6.

[44] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 4.

[45] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 4.

[46] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 13.

[47] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 19.

[48] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 21.

[49] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 22-24.

[50] “Submitting Documents Using Real-World Data and Real-World Evidence to FDA for Drugs and Biologics”, FDA Draft Guidance, May 2019, 3-4, https://www.fda.gov/media/124795/download.

[51] U.S. Food & Drug Administration, Framework for FDA’s Real-World, 9.

[52] “FDA Approves NovoSeven® RT for the Treatment of Glanzmann’s Thrombasthenia (GT) With Refractoriness,” Cision PR Newswire, July 7, 2014, https://www.prnewswire.com/news-releases/fda-approves-novoseven-rt-for-the-treatment-of-glanzmanns-thrombasthenia-gt-with-refractoriness-266071281.html.

[53] “FDA approval for Ibrance in men with breast cancer sets precedent for use of real-world evidence,” Pharmaceutical Technology, May 14, 2019 , https://www.pharmaceutical-technology.com/comment/real-world-evidence-in-pharma/.

[54] Sentinel Program Interim Assessment (FY 15), September 24, 2015, 22, https://www.fda.gov/media/93479/download.

[55] Tagrisso (Osimertinib), Postmarket Requirements and Commitments, U.S. Food & Drug Administration, last accessed November 18, 2020, https://www.accessdata.fda.gov/scripts/cder/pmc/index.cfm.

Update Magazine

Winter 2020

More Articles