The American Law Institute’s Draft Restatement of the Law of Consumer Contracts is a highly controversial project that has drawn sharp criticism from state attorneys general, consumer advocates, civil rights and labor organizations, and even a former ALI Vice-Chair, Elizabeth Warren, who is now a Presidential candidate. Reasonable minds might differ regarding the normative choices made by the Reporters in the Draft Restatement. Yet there should be no disagreement that a restatement must stand on an unimpeachable characterization of the underlying caselaw. Unfortunately, there are such pervasive and fundamental problems with the Reporters’ reading of the caselaw that no one can have confidence that the Draft Restatement correctly and accurately “restates” the law of consumer contracts.

Traditionally, Restatements have arrived at their characterization of the relevant caselaw through the type of qualitative close reading that is the hallmark of doctrinal legal analysis. The Draft Restatement contains a major methodological departure from past restatements, namely a reliance on a set of quantitative empirical studies of judicial decisions, aimed at distilling the state of the law and discerning trends in its development. Specifically, the cornerstone of the Draft Restatement is a set of six original empirical studies undertaken by the Reporters specially for the Draft Restatement.[1]

The reliance of a restatement on any original empirical work is itself problematic because legal scholarship in general, and the ALI restatement drafting process in particular, has yet to assimilate the norms of empirical research in the sciences and social sciences that have developed to ensure that empirical research is a reliable foundation for analytical conclusions regarding the law.   A foundational principle of all empirical research is that it should be susceptible to replication as a means of validating and verifying the findings.[2]   The ALI restatement process is not set up to handle empirical evidence; it lacks procedures for any sort of rigorous examination or testing. Instead the ALI restatement process is one of significant deference to the Reporters as experts in the field. As a result, none of the six studies on which the Draft Restatement was based had undergone peer review. Indeed, only one had been published, in the home institution journal of one of the Reporters,[3] before being relied upon as the basis for the Draft Restatement’s normative propositions.

We are all elected or life members of the American Law Institute who have been serving as Advisers to the Draft Restatement project or as part of the Members Consultative Group for the project. We have been among those concerned with the direction the Draft Restatement was taking, and we were further troubled to read in the fall of 2017 a draft version of Professor Gregory Klass’s attempt to replicate the Draft Restatement’s study of privacy policies. That replication attempt, the results of which are now published in the Yale Journal on Regulation, indicated serious problems with the Reporters’ coding (and hence reading) of the caselaw.

In order to assess the strength of Professor Klass’s findings, we attempted our own replication of the Draft Restatement’s privacy policy study. After finding that we uniformly concurred with Professor Klass’s reading, we decided to extend our replication attempt to the two largest datasets supporting the Draft Restatement, those on the enforceability of unilateral contract modifications and those on the enforceability of clickwrap assent, the results of which are now also published in the Yale Journal on Regulation.

For our replication study, we reviewed 186 cases, blind to the Reporters’ coding. We found that nearly two-thirds of the cases in the unilateral modification dataset were irrelevant to the hypothesis tested by the Reporters. The irrelevant cases included business-to-business cases, vacated and reversed decisions, a duplicate decision, and cases determined on statutory grounds specific to credit card agreements. The remaining relevant cases were atypical, almost entirely involving enforcement of arbitration clauses, express contractual clauses permitting unilateral modifications, or credit card agreements.

Likewise, we found that nearly half of the cases in the clickwrap assent dataset were irrelevant to the Reporters’ tested hypothesis. The irrelevant cases included business-to-business cases, another duplicate decision, and cases with neither contracts nor clickwrap agreements. The overwhelming majority of the relevant cases involved the sui generis contexts of enforcement of arbitration clauses or forum selection clauses. Among such limited relevant cases, we also found a materially higher rate of non-enforcement of clickwrap agreements than the Reporters found.

We believe it beyond peradventure that a draft restatement that fails to faithfully restate the law should not be adopted. Yet the ALI Council has greenlighted the Draft Restatement for a vote by the ALI’s membership—the College of Cardinals of the American legal profession—this coming spring. We do not speculate why the ALI Council would ever consider promulgating such a flawed product, but we are deeply concerned that the Council would have blessed a patently faulty Draft Restatement, which is as likely to confound as to clarify the state of the law and to taint the reputation of the ALI’s other projects.

[1] In the latest draft of the Draft Restatement, the Reporters now claim to that they also undertake a traditional analytical approach. This claim appears for the first time only after the Reporters were presented with our initial findings and those of Professor Klass’s study. All previous drafts of the Draft Restatement emphasized the empirical approach. Remarkably, the purported new approach arrives at exactly the same positions as the drafts supported by the flawed empirical studies.

[2] See, e.g., Brian Resnick, More Social Science Studies Just Failed to Replicate. Here’s Why This Is Good. Vox, Aug. 27, 2018, at https://bit.ly/2wy7DUS. In recent years, the field of psychology in particular has been shaken by a “replication crisis”—the inability of researchers to replicate the findings of many foundational studies. See, e.g., Ed Yong, Psychology’s Replication Crisis Is Running Out of Excuses, The Atlantic, Nov. 19, 2018, at https://bit.ly/2OU8Ht1.

[3] Oren Bar-Gill et al., Searching for the Common Law: The Quantitative Approach of the Restatement of Consumer Contracts, 84 Chic. L. Rev. (2017), at https://bit.ly/2TWnfvx.

—

Adam J. Levitin is the Agnes N. Williams Research Professor and Professor of Law, Georgetown University Law Center; Member, Members Consultative Group, ALI Project on the Restatement of the Law of Consumer Contracts. Nancy S. Kim is ProFlowers Distinguished Professor of Internet Studies and Professor of Law, California Western School of Law; Member, Members Consultative Group, ALI Project on the Restatement of the Law of Consumer Contracts. Christina L. Kunz is the Emerita Professor of Law, Mitchell Hamline School of Law; Member, Members Consultative Group, ALI Project on the Restatement of the Law of Consumer Contracts. Peter Linzer is a Professor of Law, University of Houston Law Center; Member, Members Consultative Group, ALI Project on the Restatement of the Law of Consumer Contracts. Patricia A. McCoy is a Professor of Law, Boston College Law School; Adviser, ALI Project on the Restatement of the Law of Consumer Contracts. Juliet M. Moringiello is the Commonwealth Professor of Business Law, Widener University Commonwealth Law School; Member, Members Consultative Group, ALI Project on the Restatement of the Law of Consumer Contracts. Elizabeth A. Renuart is Of Counsel, National Consumer Law Center, but acting in her individual capacity; Adviser, ALI Project on the Restatement of the Law of Consumer Contracts. Lauren E. Willis is a Professor of Law and William M. Rains Fellow, Loyola Law School, Los Angeles; Adviser, ALI Project on the Restatement of the Law of Consumer Contracts.

This post is part of a symposium on the Draft Restatement of the Law of Consumer Contracts. All of the posts in this symposium can be viewed here.