Product Testing, Questionable Science, and the Smallest, Big Developments in Consumer Product Litigation

By John Ewald, Shaila Rahman Diwan & Luke Bosso

In recent years, a growing number of seemingly scientific reports claim that U.S. Food and Drug Administration (FDA)-regulated products contain toxic substances. Many of these claims, however, are not properly vetted by the scientific community before they are blazoned in national headlines and discussed in congressional testimony. These reports also cause panic among the general public and lead to costly litigation, even if the testing is ultimately determined to be nothing more than proverbial “junk science.”^[1]

By way of background, analytical technology is always advancing. These advancements, coupled with unexpected findings across a variety of products, have increased pressure on regulators to issue new guidance and take novel action related to quality monitoring of products marketed in the United States. For instance, in 2021, the Biden–Harris Administration took significant steps to investigate and address per- and polyfluoroalkyl substances (PFAS) levels in foods and various consumer products, which included intradepartmental coordination by agencies such as FDA and the Environmental Protection Agency.^[2] Throughout 2022, FDA took a series of actions to address benzene levels in a variety of drugs and consumer products.^[3] In the beginning of 2023, FDA released its “Closer to Zero” action plan to identify mitigation strategies to lower levels of naturally occurring heavy metals in foods eaten by babies and young children.^[4]

The issue, however, is that even if companies fully comply with regulatory expectations for quality monitoring, they may still face lawsuits alleging that the manufacturers’ products are hazardous because they contain very low levels of allegedly toxic substances. Indeed, courts have experienced an uptick in mass tort and class action litigation involving foods and food packaging, drugs, supplements, and an array of consumer products ranging from underwear to cosmetics—all alleged to contain trace substances, such as benzene, PFAS, and compounds that claimants argue are dangerous.

The increasing popularity of this type of litigation is due, in part, to advancements in analytical technology that allegedly allow researchers to identify trace levels of various substances at concentrations, which were ostensibly not possible in the past. The use of analytical technology, however, is not without fault. Litigants often hire experts who use questionable analytical methods that may lead to false positive detections or—if the substance is present—bias the results toward higher concentrations. The methodological shortcomings are further compounded when experts then extrapolate from the results to infer that a product causes adverse human health outcomes.

This article both identifies potential methodological shortcomings employed by litigation experts during analytical testing and offers strategies to challenge that testing leading up to and throughout the trial. The first part of this article addresses ways to challenge the use of novel analytical methods. The second part discusses common issues from inferring that a product causes harm by containing a trace substance. Lastly, this article provides practical tips companies can use to defend against litigation-driven testing and questionable science. While many features of this article are geared towards defending against claims that a product contains impurities, the same principles can be used by claimants wishing to challenge a defendant’s own litigation testing.

I. Confirming Reliability of the Analytical Testing

Is the substance really there? And if so, how much is there? These types of questions, while relatively simple, cause significant disputes in litigation because the test methods used to answer these questions may be unreliable. Indeed, questionable scientific reports claiming that products contain trace substances are causing courts to face novel issues in evaluating the reliability, and thus the admissibility, of new analytical methods. Although analytical testing has been used in litigation for decades, limited guidance exists related to how courts can confirm the reliability of the testing. Often, the go-to resource for judges to understand complex science, the Third Edition of the Reference Manual for Scientific Evidence, only briefly mentions analytical testing within the chapter, “Exposure Sciences,” and focuses on traditional toxic tort issues where validated test methods are already well-established, such as testing for heavy metals in environmental media.^[5] Notably, the Reference Manual does not describe issues with the development and validation of new analytical methods. Additionally, while regulatory bodies and independent scientific organizations have established guidelines for validating analytical methods, these guidelines may be general and flexible in application. In turn, the lack of concrete guidance can make it difficult to challenge new test methods developed solely for purposes of litigation.

There are a number of issues that arise when considering the reliability of the analytical method. The following is a non-exhaustive list of issues that commonly arise during litigation involving trace substances. This list is not intended to provide a set of criteria that, if fulfilled, would be sufficient to admit test results. Rather, this list identifies independent reasons to potentially exclude test results.

1. General Acceptance in the Scientific Community. Courts routinely consider whether methods are generally accepted within the relevant scientific community.^[6] In many cases, established methods used during routine quality monitoring may not be sensitive enough to detect trace-level substances. Moreover, even where validated, extremely sensitive methods exist, litigation experts may nonetheless reject these methods, develop their own, and claim—without adequate basis—that their methods are improvements of the prior methods. In many jurisdictions, the development of new methods solely for use in litigation, in combination with a lack of peer-review, can render any results without any indicia of reliability.^[7]
2. Use of Standard Operating Procedures and Protocols. The Reference Manual suggests that courts consider the use of standard operating procedures when assessing the reliability of laboratory work.^[8] This is consistent with the fact that courts have often recognized the importance of using protocols in scientific disciplines that are prone to data dredging or unacceptable levels of manipulation. Within laboratory work, “standard operating procedures” typically refer to universal guidelines that every test within the laboratory should follow. “Protocols” provide similar guidance but are tailored to a specific test method. Therefore, one should consider compliance with both the laboratory’s standard operating procedures and the specific protocols used to evaluate the reliability of the testing.
3. Ability to Measure the Substance at Issue. Courts should consider whether and how analytical methods are able to measure the substance at issue. Referred to as a test method’s “selectivity” (or sometimes “specificity”), this criterion can be critical to understand. For nearly all types of analytical testing, even if a machine does a substantial portion of the work, an expert will ultimately use some level of subjective judgment to interpret whether the results are measuring the substance at issue. For some types of testing, such as scanning electronic microscopy (SEM), the raw data may only produce images that must be interpreted. In these instances, the definition of what qualifies as the substance at issue may be ambiguous and malleable, providing the opportunity for manipulation. For other types of testing, such as chromatography-based methods, a machine may be able to provide numeric results about whether the substance at issue is present, but ultimate interpretation about whether any substances may be interfering with the reading, i.e., confounding the result, is still left to the subjective interpretation of the analyst. Understanding the level of discretion utilized in testing can be important for evaluating an adversary’s test data.
4. Measurement of Surrogate Substances as a Proxy for the Analyte at Issue. In some cases, litigation experts may choose to measure a proxy substance instead of the true substance—or substances—at issue. For example, PFAS are a class of chemicals with hundreds of distinct chemical identities. In lieu of developing methods suitable for each PFAS, litigation experts may instead look for the presence of organic fluorine as a surrogate indicator that PFAS are present. These methods, however, have significant limitations that may undermine the reliability of an expert’s opinion depending on how the expert tries to use the data.
5. False Comparisons to Regulatory Investigations. Litigation experts may misapply concepts used in regulatory testing programs in ways that increase the amount of substance measured in the product. Take, for example, the utility of testing expired products. For regulatory purposes, companies may test expired products because if a product contains an acceptable level of impurities upon expiry, there is a reasonable expectation that the levels were also acceptable earlier during the product’s shelf-life. In this way, companies test expired products for a limited purpose. Litigation experts may also test expired products but then use the results in inapposite ways, such as estimating patient exposure. When challenged, the experts argue that testing expired products is standard practice but fail to acknowledge that the purposes are materially different.
6. Reporting Results in Misleading Terms. Reporting results in the proper context is critical for communicating with both judges and juries. Analytical measurements are typically reported as concentrations.^[9] At trace levels, these concentrations are often parts-per-million (ppm) or parts-per-billion (ppb). For example, 1 ppm means that the substance being measured makes up one-millionth of the total sample by weight or 0.000001%. While this should underscore that the substance is present at an extremely low concentration, litigation experts may translate analytical results into terms that strip the values of their context in ways that can be misleading. For instance, in talc litigation, rather than reporting that asbestos is present at ppm levels, experts may quantify in terms of “asbestos structures per gram,” which can cause corresponding numbers to appear more jarring to a lay audience. This is essentially the difference of reporting that a substance is present at a concentration of, for example, 0.000001% versus, for example, 20,000 asbestos structures/gram.
7. Conflating Worst-Case Scenarios with Reasonable Probability. Even once analytical testing exists, further statistical extrapolation is necessary to estimate total exposure. There are, however, a host of questionable assumptions that underlie these methods. For instance, litigation experts may infer levels of the substance in historical products that were never tested. Also, experts may opt to extrapolate from worst-case scenarios, even if those values are outliers and unrepresentative of true probability.
8. Misapplication of Detection Limits. Scientists are limited in the ability to confirm the absence of substance.^[10] This is because there is always the possibility that substance is present at concentrations lower than what can be detected by the analytical technology. Therefore, the absence of a finding is generally reported as “not detected” or “below limit of detection”—as opposed to a true zero—as a matter of scientific convention.^[11] Litigation experts may misuse the convention to support speculation that: 1) all samples have the substance, but 2) the technology is only sensitive enough to detect the substance in some of the samples. This reasoning attempts to shift the burden on companies to prove that a substance is not present—a scientific impossibility.

II. Confirming Reliability of the Causation Analysis

Even if a substance is present, there is not necessarily any realistic risk to human health. This illustrates an important point that analytical testing alone would be insufficient to carry the claimant’s burden of proof. As discussed further below, claimants would also need admissible expert testimony establishing that exposure to the product actually caused their injuries. The process of trying to prove causation based on exposure to a product containing trace substances presents further opportunity for litigation experts to engage in questionable scientific reasoning. While not exhaustive, the below list provides some examples of how litigation experts may misconstrue scientific concepts, particularly analytical data, when formulating their causation opinions.

1. Reliance on Hazards Instead of Risks. The difference between hazards and risks is critical. As the Reference Manual explains, all substances are intrinsically hazardous because at high enough doses, all substances, including water, can be toxic.^[12] The concept of risk, however, considers whether an adverse outcome is likely to occur at a certain dose. Information on the hazards of a substance provides no value when the levels are so low that there is no risk. Therefore, epidemiological data evaluating the specific product at issue—and de facto, any trace substances that may be in those products—is needed to evaluate true risk. In the absence of such evidence, essentially all reputable medical and scientific organizations would consider the evidence insufficient to determine that the product causes the alleged outcome. Nonetheless, litigation experts may resort to lesser forms of evidence, such as high-dose animal studies or in vitro toxicological assays. These lines of evidence, however, can only support a potential hazard and cannot reliably establish risk in humans at low doses.
2. Assumptions That No Level is Safe. To avoid evidentiary burdens of establishing that a particular claimant was exposed to sufficient doses to cause an adverse event, some claimants argue that there is no safe level of exposure. These types of theories assume there are no practical thresholds before toxicity may occur.^[13] The issue claimants run into, however, is that many substances are ubiquitous, either because they occur naturally in the environment or because of the prevalence of their use across manufacturing industries. Therefore, all people are exposed to some level of the substance at issue, regardless of whether they use the product, which calls into question the viability of no-threshold theories. Faced with tension between real-world exposure and assumptions of no-safe-exposure thresholds, litigation experts may retreat to an alternative position that any additional exposure from use of these products is additive to background exposure and must result in an increased risk. This reasoning, however, generally lacks support by empirical data, such as data that compares how total exposure (also referred to as body burden^[14]) modifies risk. Thus, these assumptions are merely speculative.
3. Use of Analogies to Substitute for Associations. Analogy may be used as a criterion to infer causation as one of the Bradford Hill criteria. Reasoning by analogy, however, is only ever used after establishing a consistent, statistically significant association between the specific product at issue and the specific outcome at issue when the association cannot be explained by chance, bias, or confounding.^[15] Litigation experts, however, may use analogy in lieu of finding a reliable association. For instance, as noted earlier, PFAS are a class of compounds with thousands of unique and distinct species. Small differences in molecular structure can result in significant differences in potential toxicity, potency, target organs, and other important differences, such that each PFAS species must be separately evaluated for risk. Nonetheless, litigation experts may improperly import risk information for one PFAS species onto another PFAS species by means of analogy to fill evidentiary gaps. Using analogies as a substitute for a valid association erroneously skips the first step of a causation analysis and would render an expert opinion unreliable.

III. Strategies to Combat Litigation-Driven Science

Even if methodological shortcomings in litigation testing are apparent, companies still need to develop strategies for how to distill complex science for both judges and juries. Upfront investment in a rigorous scientific defense can yield substantial litigation advantages and prevent prejudicial junk science from reaching the jury’s ears.

1. Pre-Litigation Strategies

Some of the most critical steps to combat litigation-driven science occurs before the first lawsuit is ever filed. A company can position itself for potential litigation by pressure-testing a company’s quality systems to ensure that they are well-maintained and follow best scientific practices. This can be critical because when an adversary conducts new scientific testing for purposes of litigation, a company will want the full opportunity to hold the litigation-driven testing to rigorous scientific standards. Attacks against litigation-driven science can be bolstered when the company can also point to their own internal testing that meets higher scientific standards than used in litigation. Additionally, should the case proceed to trial, educating the jury on the operation and safeguards within a company’s quality systems can be critical to mounting a persuasive defense.

In a similar vein, any conclusions drawn from a company’s investigations, including health risk assessments, should provide appropriate context that details the rationale for the investigation and any inherent limitation to those statements. Including appropriate qualifications to any scientific analysis can curb the ability for litigants to later take the analysis out of context. In litigation, a claimant may attempt to draw false comparisons between their own expert’s causation opinions and a company’s actions done out of an abundance of caution. Therefore, appropriately documenting and contextualizing a company’s investigations can be critical to mount an effective scientific defense.

2. Pre-Trial Strategies

If, and when, a company is faced with litigation-driven testing brought by an adversary, companies should seek full discovery of the underlying documentation. Most jurisdictions have rules that require automatic disclosure—or at minimum allow discovery—of an expert’s litigation testing. While the exact documents to request will depend on the type of analytical test utilized and other case-specific factors, at minimum one would need a laboratory’s standard operating procedures, all test protocols, all laboratory notebooks, all raw data, and executed chain of custody forms in order to evaluate the reliability of test results. A careful review of this documentation may elucidate critical flaws that undermine the reliability of the testing and thus erode confidence in any expert’s opinion that relies on such testing.

Moreover, even if an adversary’s analytical measurement is entirely reliable, companies should consider whether to challenge litigation testing because the expert failed to reliably interpret the results and apply them to the facts of the case. As discussed above, litigation experts may utilize a number of questionable assumptions and extrapolations that extend otherwise reliable results beyond their natural interpretation. For instance, litigation experts may test a limited set of products and then generalize those findings to all products. Very few experiments, however, can support such broad generalization. Litigation experts may also test the product under certain conditions and assume that the product would behave similarly under different conditions, even when there is no empirical data to support these assumptions. There are endless ways in which litigation experts may misuse data, and not every logical leap may be readily apparent. However, an investment in a rigorous scientific defense can elucidate the most critical issues underlying a set of experiments.

Beyond the reliability of the testing in and of itself, companies may also challenge the admissibility of testing through traditional motions in limine if the testing is not relevant for a particular trial. For instance, if a litigation expert tested products—but not a claimant’s specific product—the testing is likely irrelevant and unduly misleading. While these arguments may appear similar to challenges based on the reliability of an expert’s opinion, there may be strategic reasons to move to exclude the results on relevance grounds. Firstly, depending on the jurisdiction, the case law on relevance may be more robust and favorable. Secondly, many judges may feel more confident in excluding an expert based solely on legal grounds, such as relevance, as opposed to weighing in on scientific methodology.

3. Trial Strategies

Companies facing a prospective jury trial risk that the potential jurors adopt a “zero tolerance” attitude to trace substances in products, especially in absence of evidence that the product is lifesaving or bestows justifiable benefit. This attitude can lead jurors to assume the company may have acted recklessly, despite evidence that a company maintained quality systems. Jurors may also be tempted to disregard scientific evidence beyond analytical testing, even if the evidence overwhelmingly reinforces the product’s safety. Therefore, companies facing a prospective jury trial should consider strategies to effectively defend the company’s reputation and quality processes.

First, perspective is everything. Depending on the case, people are likely to be exposed to the exact substance at issue on a regular basis. If applicable, comparing background exposure to the doses in a product can provide the jury with much-needed perspective that the product contributes, at most, de minimis additional exposure above background. Working with graphic consultants can prove useful to develop accessible demonstratives that show a product’s contribution, if any, is dwarfed by the overall background exposure.

Second, a critical trial strategy dates back to the sixteenth century when the philosopher Paracelsus first articulated a central tenet of toxicology: “the dose makes the poison.”^[16] This phrase refers to the concept that even hazardous substances are unlikely to cause adverse effects at low doses. Because trace substances are, by definition, present in infinitesimally small levels, comparing the doses that result in appreciable increased risk versus the levels present in a product can elucidate evidentiary failures in proving causation. Companies should consider targeted strategies to develop these accessible, jury-friendly admissions through the opposing expert witnesses. These themes should be incorporated into overall deposition strategy so that the company can establish the necessary admissions for an effective cross examination at trial.

Third, telling the story of a company’s due diligence cannot take a backseat and should always be paired with the scientific evidence. In fact, explaining the reality surrounding how scientists identify and control trace substances can be critical for the jury to contextualize the analytical findings. Jurors often begin with the misconception that the presence of a trace substance is an atypical occurrence and that it is evidence of a quality failure. These assumptions, however, fail to acknowledge that no substance is truly “pure.” And since no product or material is truly pure, the pertinent question is whether the presence of the newly discovered substance was reasonably foreseeable. Of course, foreseeability always seems easier in hindsight, so a company should develop strategies for how to leverage its manufacturing and quality control practices affirmatively in its defense.

Finally, the burden of proof is key. As described above, an adversary may rely on arguments such as assuming that there is no safe level of exposure to a substance, assuming that a non-detect test result could mean the substance is nevertheless present, or making leaps of logic from high-dose animal studies to lower doses in humans. An adversary’s reliance on these opinions should be to their own detriment if that party carries the burden of proof.

IV. Conclusion

Trace substance litigations are rising in popularity, and all manufacturers should begin to consider how best to position themselves for potential litigation in the event new substances are discovered in their products. While specific strategies vary based on the needs of the litigation, there is a common set of strategies for companies to apply both before and across varying litigations.

 

[1] U.S. Dep’t of Justice, Report of the Tort Policy Working Group on the Causes, Extent and Policy Implications of the Current Crisis in Insurance Availability and Affordability (Feb. 1986).

[2] The White House, FACT SHEET: Biden-⁠Harris Administration Launches Plan to Combat PFAS Pollution (Oct. 18, 2021), https://www.whitehouse.gov/briefing-room/statements-releases/2021/10/18/fact-sheet-biden-harris-administration-launches-plan-to-combat-pfas-pollution/?utm_source=link.

[3] E.g., U.S. Food & Drug Admin., FDA Alerts Drug Manufacturers to the Risk of Benzene Contamination in Certain Drugs (current as of Dec. 23, 2022), https://www.fda.gov/drugs/pharmaceutical-quality-resources/fda-alerts-drug-manufacturers-risk-benzene-contamination-certain-drugs.

[4] U.S. Food & Drug Admin., Closer to Zero: Action Plan for Baby Foods (current as of Jan. 12, 2023), https://www.fda.gov/food/environmental-contaminants-food/closer-zero-action-plan-baby-foods.

[5] Reference Manual for Scientific Evidence, 3rd Ed. at 528–30.

[6] Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 594 (1993).

[7] See id.

[8] Reference Manual for Scientific Evidence, 3rd Ed. at 529–30.

[9] Id. at 541.

[10] Id. at 530.

[11] Id. at 530.

[12] Id. at 636.

[13] Id. at 642.

[14] Id. at 537.

[15] Id. at 598–99.

[16] Id. at 603, n. 160.