BYOD and eCOA: A Match Made in Heaven?
by Frederick Ball, Carolyn Alenci, Sandra Stoneman
Patient-focused drug development and the selection and development of Clinical Outcome Assessments (COA) continue to be a focus for the U.S. Food and Drug Administration (FDA). At the same time, we continue to see an increase in technology available at our fingertips and on our wrists. As electronic capture of data becomes more robust and systems to ensure its integrity are put into place, FDA has started to embrace electronic clinical outcome assessments (eCOA). This increase opens up a plethora of new data sources that can be used to facilitate and enhance clinical trials, including the use of a study subject’s own devices (a/k/a “bring your own device” (BYOD)). This article discusses eCOA, BYOD, and FDA’s guidance on their use in clinical studies.
Advantages of eCOAs
Like all COAs, eCOAs must be “fit-for-purpose” and must provide well-defined and reliable measurements of a patient’s symptoms, overall mental state, ability to function with the disease, or condition being studied. For any COA, FDA suggests that sponsors should plan for early interaction with FDA for feedback on their COA strategy through the medical product development program, COA qualification, or by seeking general advice through critical path innovation meetings, pre-submission meetings, or other meetings with FDA.1 There are essentially five types of eCOA systems, as follows: (1) interactive voice responses; (2) web-based platforms; (3) mobile applications and wearable sensors; (4) digital pens; and (5) tablets. Some of the possible advantages of eCOAs include:
• no need for manual secondary data entry;
• direct transmission into an electronic database;
• alarm or reminder capabilities;
• time and date stamp capabilities;
• real-time data recording and transmission; and
• remote data capture. 2
Essentially, eCOAs can produce highly accurate data that is generated in “real-time” and can be easily attributable to a subject. eCOAs also allow for less room for misinterpretation of data than handwritten notes in traditional COAs. Further, programming daily reminders using an electronic device’s alarm
function, along with external alarm methods (e.g., email, phone call, and/or text alerts), minimizes the amount of missing data and allows for automatic recording of important information like timestamps.3
FDA Finds Love for eCOA
FDA’s evaluation of a study sponsor’s COA involves three prongs:
(1) understanding the disease or condition that is the subject of the study;
(2) conceptualization of clinical benefit of the treatment; and
(3) selecting, developing, or modifying the COA.
More specifically, FDA will make its determination of a COA’s “fit-for-purpose” by reviewing whether the COA is appropriate for its intended use (e.g., study design, patient population) and whether the COA validly and reliably measures concepts that are clinically relevant and important to patients.
Finally, FDA will determine whether the COA data can be communicated in a way that is accurate, interpretable, and not misleading (i.e., well-defined).4 Importantly, specifically with respect to eCOAs, FDA requires that eCOAs undergo rigorous validation processes prior to their use in order to ensure device and program functionality and performance stability within the clinical trial context. FDA also recommends that study sponsors maintain a back-up plan for when there are malfunctions with the electronic devices.
Examples could include web-, phone-, or paper-based methods for capturing the data. Likewise, sponsors should provide FDA with device usability testing analysis results, electronic screenshots of the instruments, training materials, and documentation related to migrating or reformatting to electronic format.5 Moreover, eCOA data must meet all regulatory guidelines and considerations, including all requirements for recordkeeping, maintenance, and access, as well as the requirements of the predicate rules.6
This includes compliance with 21 CFR Part 11 with respect to electronic records and electronic signatures.7 FDA also requires that all electronically authorized data originators be maintained at each clinical site and employ measures to ensure security and integrity of the authorized users and passwords. Electronic data originators can be an individual person or an automated system. Examples include:
• clinical investigator(s) and delegated clinical study staff;
• clinical investigation subjects or their legally authorized representatives;
• consulting services (e.g., a radiologist reporting on a computed tomography (CT) scan);
• medical devices (e.g., electrocardiograph (ECG) machine and other medical instruments such as a blood pressure machine);
• electronic health records (EHRs);
• automated laboratory reporting systems (e.g., from central laboratories); and
• other technology. 8
All that being said, FDA understands the importance of actual patient input in product development. In addition to the standard data collected in clinical studies, actual real-world patient input is critical in helping ensure safe and effective treatments and in informing the benefit-risk assessments during FDA’s decision-making process.9 With COA enabling the recordation and collection of this patient input, it is our view that eCOA is here to stay and only likely to get increased support from FDA as the accuracy and strength of electronic data collection becomes more solid.
Are BYOD and eCOA a Match?
The use of BYOD to collect self-reported outcomes in eCOA or electronic patient-reported outcomes (ePRO) is garnering significant interest in the industry.10 There are many reasons that sponsors may wish to include a BYOD option in their protocols. Protocols that include BYOD generally allow clinical trial subjects to use their personal devices to access and respond to study-related questionnaires. In some cases, sponsors will use web-enabled devices, such as smartphones, to allow subjects to access questionnaires over a web browser, while others may require that the subject download an app to access the questionnaires. The use of BYOD can reduce costs for the sponsor and reduce burdens on the patients and the study sites.
Over the last couple of years, FDA has sought comments on the use of BYOD technology in clinical research in an effort to assess these comments relative to its own concerns. Not surprisingly, some of FDA’s concerns with BYOD are the same concerns it has for eCOA, including establishing “fit for purpose,” ensuring user authentication, incorporating medical devices, failure modes, and clinical monitoring. However, other concerns are unique to BYOD.
Nevertheless, because of the potential cost savings and ease of use, BYOD in clinical trials will likely be on the upswing. Concerns regarding data “equivalency” are paramount to FDA when evaluating BYOD. Therefore, study sponsors will need to show that instruments being used have been validated to produce equivalent data and will need to perform qualitative equivalence studies when using mixed modes. Study sponsors will also need to demonstrate that clinical investigators will have access to data, that vendors are qualified, and that the time to progression endpoints is reasonable. Study sponsors should consider the fact that they will not have control of the device, nor of the storage available on the device, including no control over upgrades to operating systems, activation/inactivation of widgets, and other apps that might be present that could impact the study sponsor’s app. 11
Another matter to consider is that when study sponsors use BYOD, they should have a back-up device option for those without their own devices.12 In other words, sponsors need to decide whether to make ownership of a device part of the inclusion criteria or whether it will provide participants with standalone hardware or a backup device if they do not have the appropriate device.13 Sponsors also need to reduce variations in instrument format and functionality through the use of a single platform during clinical trials.14
Ultimately, like all eCOAs, if a sponsor chooses BYOD, it must present FDA with a detailed plan, including security (e.g., data storage, data transfer, subject’s privacy, user authentication), compatibility (e.g. iOS vs. Android), what data will be accessed from the subject’s device, contingencies for when capturing data even when a subject deletes the app, data connectivity, accessibility of data to clinical investigator, data availability upon FDA inspection, clinical monitoring, failure modes, how to make sure subjects opt-in for notifications, and subject training. 15 This plan should be presented to FDA early in the development of study protocols.16
The Marriage of eCOA and BYOD in the Eyes of FDA
FDA continues to support the use of eCOA and notes a number of advantages to its use, including the lack of manual secondary data entry by another person, reduction in risk to data integrity, ability to provide reminders to subjects, and minimization of missing data, among others. FDA is likely to continue to view eCOA favorably and encourage sponsors to make use of eCOA in their studies because of the positive effect eCOA can have on quality and integrity of data submitted when seeking approval of a new drug, biosimilar, device, or new indications for an already approved medical product. For the same reasons, FDA is likely to favor the use of eCOA in post-market studies. The risk/benefit of BYOD as part of eCOA is less clear, and, therefore, it is more challenging to predict FDA’s future views on this matter. On the one hand, the FDA is likely to view the hacking, risk to privacy, and potential data integrity issues with BYOD as problematic. On the other hand, while devices provided by a sponsor may be more secure, at least in theory, there are other risks (e.g., recall risks due to device malfunctions such as over-heating). Robust testing and validation for sponsors wishing to use a BYOD model in order to minimize the likelihood of the potential issues is likely going to be required. And FDA will likely closely monitor studies where BYOD is used, which in turn could affect FDA’s view on the BYOD in the future. The additional safeguards that need to be imposed to assure data integrity may also be costs that do not outweigh the benefits of a BYOD model. Therefore, the BYOD and eCOA need a longer courtship before we will know if their match is really a “match.”
1. FDA, Patient-Focused Drug Development Guidance Public Workshop: Methods to Identify What is Important to Patients & Select, Develop or Modify Fit-forPurpose Clinical Outcome Assessments (Oct. 15–16, 2018), at n.4, lines 248–50, 279–80, 296–302; FDA, Clinical Outcome Assessment Qualification Program,available at https://www.fda.gov/drugs/developmentapprovalprocess/drugdevelopmenttoolsqualificationprogram/ucm284077.html (last updated May 25, 2018) (internal citations omitted).
2. Patient-Focused Drug Development Guidance Public Workshop, supra note 1 at lines 1304–28, 1336–40 (citing Christine Hall, Comparing the Five Methods to Collect Patient-Driven eData, Applied Clinical Trials (Sept. 26, 2013)).
3. Id. at lines 1341–45; see also FDA, Guidance for Industry: Electronic Source Data in Clinical Investigations (Sept. 2013), at 4–6.
4. Id. at lines 267–70, 280–86.
5. Id. at lines 1346–49, 1394–96, 1398–1402, 1406–08.
6. Patient-Focused Drug Development Guidance Public Workshop, supra note 1 at lines 1421–24, 1426–28.
7. Id. at n.17; see also, FDA, Use of Electronic Records and Electronic Signatures in Clinical Investigations Under 21 CFR Part 11—Questions and Answers: Guidance for Industry (June 2017) (issued as draft guidance).
8. Electronic Source Data in Clinical Investigations, supra note 3 at 3–4, 8.
9. Patient-Focused Drug Development Guidance Public Workshop, supra note 1 at lines 218–31.
10. Id. at lines 1351–52; see also Fifth Annual Patient-Reported Outcome (PRO) Consortium Workshop, Session 8: Bring Your Own Devices (BYOD) Application in Clinical Trials, at 7 (Apr. 29–30, 2014), available at https://c-path.org/5th-annual-patient-reported-outcome-pro-consortium-workshop/.
11. Id. at 7–9, 12, 21.
12. Patient-Focused Drug Development Guidance Public Workshop, supra note 1 at line 1352.
13. Fifth Annual Patient-Reported Outcome (PRO) Consortium Workshop, supra note 10 at 10.
14. Patient-Focused Drug Development Guidance Public Workshop, supra note 1 at lines 1352–55.
15. Id. at lines 1355–57; see also, e.g., Fifth Annual Patient-Reported Outcome (PRO) Consortium Workshop, supra note 10 at 12, 15, 16, 20, 21, 26, 28.
16. See Patient-Focused Drug Development Guidance Public Workshop, supra note 1 at n.4, lines 248–50, 279–80, 296–302; Clinical Outcome Assessment Qualification Program, supra note 1