The draft Restatement of the Law of Consumer Contracts invokes six quantitative studies of judicial decisions. Each study seeks to collect all available decisions on a legal question, published and unpublished; codes those decisions for factors such as issue, outcome, procedural posture, jurisdiction, and citations; and analyzes the coded data to determine majority rules, trends, and lines of influence. This Article reports the outcome of an attempt to reproduce the numerical results of the Reporters’ study of whether courts treat business privacy policies as part of their contracts with consumers.
There are four takeaways. First, this study finds that the Reporters’ data regarding the judicial treatment of privacy policies do not adequately support the conclusions they draw or the proposed comment. Second, the results of this study, especially together with those of Levitin, et al., suggest that the draft Restatement’s other quantitative case law studies should not be considered definitive until their methods have been examined and their results reproduced. Third, this study’s results illustrate the importance of transparency and replication in quantitative empirical studies. This holds all the more for attempts to quantify judicial decisions, whose coding often requires difficult judgment calls about both meaning and legal effect. Finally, part of the problem in this instance might be the American Law Institute’s procedures, which are not designed for transparency or to give outside scholars an opportunity to examine or attempt to reproduce empirical results. If the ALI wishes to use quantitative case law studies in future Restatements, it should consider revising those procedures.
Gregory Klass is a Professor of Law at Georgetown Law. This post is based on an article from the latest print edition of the Yale Journal on Regulation