*This post is part of a symposium on Modernizing Regulatory Review. For other posts in the series, click here.

Circular A-4 should be revised to recognize the essentially stochastic nature of cost-benefit analysis. This is not just a theoretical concern, but one that has far-reaching policy implications. Agencies often must regulate in the face of deep uncertainty—more precisely, risk—regarding the future. This type of uncertainty is orthogonal to measurement uncertainty (which Circular A-4 does discuss at length). Instead, it concerns the risk regarding the state of the world that would materialize if the proposed rule or policy were to be adopted. Even when an agency considers a rule’s potential benefits and costs with utmost care, there is an inherent randomness to how the economy will evolve through a spontaneous order after the rule is implemented. The notice-and-comment process can typically help the agency discern these possibilities prior to rule adoption, and the agency should structure its cost-benefit analysis to acknowledge the possibility of multiple states of the world. 

Once we acknowledge the stochastic nature of cost-benefit analysis, several important policy implications follow, all of which should be discussed in Circular A-4.

First, the net benefit of an agency rule is best characterized as a random variable whose possible values are outcomes of a random process. This implies, for example, that the net benefit of a regulation should be discussed in terms of its mean (expected value) and variance. It also implies that a discrepancy between the realized value of the regulation and its expected value does not necessarily indicate that the agency miscalculated the expected value.

Second, policy reversibility—that is, the ease with which the agency can revert back to status quo—ought to be a key consideration in any regulatory analysis where there are genuine disagreements about the potential effect of the rule. Specifically, all else equal, a regulatory approach that is reversible should be favored over one that is not even if the reversible approach presents the same (or even lower) expected value. In practice, it may make sense to consider reversibility as falling on a spectrum (rather than as a binary concept) and thus, it would make sense to consider rule stickiness and the cost of reversion. 

Third, a regulatory approach that comes with one or more built-in mechanisms for future rule revision—such as a sunset clause or conditional ex post exemptive relief—should be favored over one without. The possibility of future revision allows for experimentation and regulatory learning. Accordingly, option theory would suggest that, in the face of deep uncertainty, agencies should take a real-options approach to evaluating their policy choices. In particular, Circular A-4’s guidance to agencies should clarify that, in conducting cost-benefit analysis, agencies should consider including the option value of repealing or modifying the policy in the future if the chosen policy were to prove to be inefficient. 

Fourth, variance matters. When two reversible policy choices present similar or identical net expected benefits, the choice that exhibits a higher variance ought to be favored because this implies a higher upside benefit (with the understanding that the rule would be repealed if an inefficient state were to be realized). In particular, under a real-options approach, an agency could be justified in adopting the policy choice with a lower mean if the rule comes with a higher variance. 

The rest of this post will include a review of select academic articles that have highlighted these ideas. The bibliography is also included at the end.  

One of the earliest thinkers on the role and the scope of government planning was Friedrich Hayek. Hayek was among the first to recognize the essentially stochastic nature of government policy outcomes. Cautioning against a sweeping program to construct a rational order to arrange the economy, Hayek noted in 1965 as follows: “The crucial fact of our lives is, however, that we are not omniscient, that we have from moment to moment to adjust ourselves to new facts which we have not known before, and that we can therefore not order our lives according to a preconceived detailed plan in which every particular action is beforehand rationally adjusted to every other.” (Hayek, 1965, p.7). Consequently, Hayek was skeptical of the view that “we have it in our power so to shape our institutions that of all possible sets of results that which we prefer to all others will be realized.” (Id.). Instead, he emphasized the “nature of spontaneous order” and relayed the importance of improving “abstract rules” with which individuals and institutions ought to respond to unforeseen circumstances.” (See id. pp.8-9).

A more explicit recognition that policy outcomes are inevitably stochastic comes from Allen V. Kneese’s 1968 manuscript, which motivated a response from Kenneth J. Arrow and Anthony C. Fisher. In their 1974 article, Arrow and Fisher set out to explore “the implications of uncertainty surrounding estimates of the environmental costs of some economic activities.” (Arrow & Fisher, 1974, p.312). They begin by referring to Kneese’s critique of the analyses conducted by several economists regarding environmental policies: Kneese observed that “a general shortcoming of [these studies] has been that they have treated a stochastic or probabilistic phenomenon as being deterministic.” (See id.(emphasis added)). Arrow and Fisher acknowledge that “[a]ny discussion of public policy in the face of uncertainty must come to grips with the problem of determining an appropriate attitude toward risk on the part of the policy maker” and take up Kneese’s inquiry: “Is the concept of mathematical expectation applicable here or must we give attention to higher moments of the probability distribution . . . ?” The authors’ policy prescription is that “the expected benefit of an irreversible decision should be adjusted to reflect the loss of options it entails.” (Id. p.319). Arrow and Fisher thus propose that, given the stochastic nature of policy outcomes, the government ought to apply option theory in policymaking.

Cass Sunstein, who formerly served as OIRA Administrator himself, has suggested a few basic approaches in which regulators can apply Arrow & Fisher (1974)’s options approach to environmental regulation—specifically, with respect to irreversible or catastrophic harms. Sunstein argues that “[w]hen a harm is irreversible, and when regulators lack information about its magnitude and likelihood, they should purchase an ‘option’ to prevent the harm at a later date—the Irreversible Harm Precautionary Principle.” (Sunstein, 2005, p.841). He remarks that such a “principle would bring standard option theory to bear on environmental law and risk regulation.” (Id.). While Sunstein focuses on the irreversibility of natural harms—rather than policy irreversibility—his recommended application of option theory to agency policymaking is significant because it counsels that the government ought to approach irreversible outcomes in a markedly different manner from reversible outcomes. In a later piece, Sunstein also remarks that “[i]t is noteworthy, and highly revealing, that the precautionary principle does not appear in the governing executive orders; cost-benefit balancing is endorsed in.” (Sunstein, 2014, p. 181).

Listokin (2008) focuses extensively on the difference between reversible regulations and irreversible regulations in the realm of policymaking. Noting that “[t]he best risk regulation from an optimal search perspective constitutes a modification of cost-benefit analysis that contrasts with the precautionary principle even more sharply than conventional cost-benefit analysis,” Listokin proposes applying the real-options approach to agency rulemaking. (Id. p. 485). Consequently, he notes that high variance policies should be preferred all else equal. He also illustrates that, given two reversible policy choices, one with a lower mean could be preferred if it has a higher variance. Towards the end of the article, Listokin notes that “the desirability of high-variance/low-expected-value regulations depends critically upon where along [the] reversibility spectrum the regulation falls.” (Id. p. 546). As a result, he explicitly calls for a revision of Circular A-4: “the informational value of a policy that would be analyzed under the OMB’s Circular No. A-4 should include an examination of the potential persistence of a regulation’s effects.” (Id.).

Lee (2013) takes one step further. Writing in response to the D.C. Circuit’s exacting standard of review in Business Roundtable v. SEC, 647 F.3d 1144 (D.C. Cir. 2011), Lee argues that building mechanisms for future rule revision into a rule —such as a sunset clause or conditional ex post exemptive relief—must be viewed as increasing the option value of the rule. Three conclusions follow from Lee (2013): first, an agency choosing to take such a route should explicitly include this option value in its cost-benefit analysis; second, an agency engaging in a rulemaking, particularly when the proposed rule generates significant disagreements among commenters regarding the rule’s likely effect, should carefully consider incorporating such mechanisms to enhance the rule’s overall expected value; and third, courts should be extra deferential to agency rules that come with such mechanisms given the value such rules provide in terms of experimentation and regulatory learning. 

Romano (2014) also highlights the unknowability of policy outcomes. Writing specifically in the context of financial regulation, Romano makes the following observation: “financial firms operate in a dynamic environment in which there are many unknowns and unknowables,” and as such, “even the most informed regulatory response . . . will be prone to error, and is likely to produce backward-looking regulation that takes aim at yesterday’s perceived problem, rather than tomorrow’s . . . .” (Romano, 2014, p.28). She has thus argued for “sunset provisions requiring subsequent review . . . , along with regulatory exemptive or waiver powers that create flexibility in implementation and encourage . . . small-scale, discrete experimentation to better inform and calibrate the regulatory apparatus.” (Id.). In a later work, Romano would go on to elaborate on this idea of sunsetting as an “adaptive strategy.” See Romano & Levin (2021). 

Spitzer & Talley (2014) raise similar points as Lee (2013). They interpret Business Roundtable as a form of “judicial skepticism toward experimentation (and the real option to abandon) in the [cost-benefit analysis] calculus” and argue that “agencies have arguably been discouraged from counting as a benefit the value of information obtained through adopting new regulations on a provisional basis, with an option to revert to the status quo in the future.” (Spitzer & Talley, 2014, p. S121). After carefully considering the value of field experimentation and that of conventional cost-benefit analysis, they “demonstrate that there is no principled basis for dismissing (or demoting) experimentalism and that such rationales deserve a place in agencies’ standard CBA arsenals.” (Id.).  For a more extended discussion on regulatory experimentation, see Gubler (2014) and Gubler (2015).

Lee (2023) builds on Hayek (1965) and Romano (2014) to formalize the idea that cost-benefit analysis ought to be viewed as a random variable. Lee notes as follows: 

[W]hen the future state can only be ascertained probabilistically over multiple possible states, the economic value of a given regulation will itself be a random variable, and the agency’s calculation of the net benefit will be akin to ascertaining the variable’s mean.  If the variable comes with a high variance, the mean will bear no meaningful relation to any realized value. As such, the distance between the realized value and the prediction should reveal nothing about whether the regulator’s calculation of the mean was correct, just as rolling a 1 with a fair die does not undermine the fact that its expected value was (and still is) 3.5. What the distance would reveal instead is a need to reassess and adapt to the realized outcome. (Lee 2023).

In addition, Lee catalogs a variety of factors that can introduce randomness to policy outcomes. These include strategic behavior, the problem of multiple equilibria, ecological rationality, and cognitive biases. As a result, Lee proposes a new way of understanding notice-and-comment rulemaking: “[APA] Section 553 should be viewed not as a deterministic process for identifying the ‘correct’ regulatory solution to a given problem, but as a deliberative process for agreeing upon a reasonable first step that can trigger an adaptive process for addressing the problem in gradual steps.” (Id.).

Although this is only a brief review, the implications are clear: Circular A-4 should be revised to recognize the essentially stochastic nature of cost-benefit analysis. Circular A-4 currently includes no meaningful discussion on policy uncertainty and policy reversibility and this ought to change. Cost-benefit analysis must not be understood as a deterministic endeavor subject only to measurement or quantification errors. Circular A-4 should be revised to encourage agencies to acknowledge policy uncertainty and to carefully analyze—in addition to cost-benefit analysis—the reversibility of the rule they choose to adopt. Circular A-4 should clarify that, all else equal, regulatory approaches that are reversible should be favored to those that are irreversible, and regulatory approaches that build in mechanisms of future rule revision should likewise be favored. 

Yoon-Ho Alex Lee is Professor of Law at Northwestern Pritzker School of Law and Director of Northwestern University Center on Law, Business, and Economics.

---

Bibliography

Arrow, Kenneth J. & Anthony C. Fisher. 1974. “Environmental Preservation, Uncertainty, and Irreversibility.” Quarterly Journal of Economics. 88:312-319.

Hayek, Friedrich. 1965. “Kinds of Rationalism.” Economic Study Quarterly. 15:1-12.

Gubler, Zachary J. 2014. “Experimental Rules.” Boston College Law Review. 55:129-79.

Gubler, Zachary J. 2015. “Making Experimental Rules Work.” Administrative Law Review. 67:551-93.

Lee, Yoon-Ho Alex. 2013. “An Options Approach to Agency Rulemaking.” Administrative Law Review. 65:881-941.

Lee, Yoon-Ho Alex. 2023. “Beyond APA Section 553: Hayek’s Two Problems and Rulemaking Innovations.” George Washington University Law Review, forthcoming.

Listokin, Yair. 2008. “Learning Through Policy Variation.” Yale Law Journal. 118:480-553.

Spizer, Matthew & Eric Talley. 2014. “On Experimentation and Real Options in Financial Regulation.” Journal of Legal Studies. 43:S121-S149.

Sunstein, Cass. 2006. “Irreversible and Catastrophic.” Cornell Law Review. 91:841-98.

Sunstein, Cass. 2014. “The Real World of Cost-Benefit Analysis: Thirty-Six Questions (and Almost As Many Answers).” Columbia Law Review. 114:167-212.

Romano, Roberta. 2014. “Regulating in the Dark and a Postscript Assessment of the Iron Law of Financial Regulation.”Hofstra Law Review 43:25-93. 

Romano, Roberta & Simon A. Levin. 2021. “Sunsetting as an Adaptive Strategy.” PNAS. 118(26):e2015258118.