Reviewing Administrative Review

In the largest system of federal adjudication—Social Security disability adjudication—outcomes depend more on the randomly assigned judge than on the strength of the case. Does the administrative appeals process use resources effectively to reduce that arbitrariness and limit the discretion of administrative law judges? If not, how and why does it fail? These are empirical questions, and this Article uses a new dataset tracking millions of cases to answer them.

A system of administrative appeals that efficiently limits the discretion of decisionmakers should display three empirical patterns. First, disappointed claimants should be more likely to appeal the decisions of harsher judges—judges who have lower grant rates than their colleagues in the same hearing office (claim selection). Second, when claimants appeal, harsher judges’ decisions should be reversed more often than the decisions of their more generous colleagues (decisionmaking). Third, judges should try to avoid remands and therefore increase their grant rates after a reversal (remand aversion).

Testing for each of these patterns offers a method of diagnosing problems with systems of administrative review—and helps identify where new resources would be most useful. For example, if litigants rarely appeal decisions of even extreme adjudicators, a quality assurance process might solve the problem by randomly selecting cases for review. If appellate decisionmaking itself is flawed, peer review may be more promising. And if adjudicators are insensitive to remands, training and feedback might be appropriate.

I apply this method to conduct the first quantitative study of administrative appeals in Social Security disability adjudications. I use a previously unreleased database, obtained by Freedom of Information Act request, of nearly four million Social Security Administrative Law Judge and Appeals Council decisions from 2010 to 2014. Social Security Appeals Council decisions do not show the expected decisionmaking pattern: on average, the Appeals Council fails to distinguish between the decisions of harsher and more generous adjudicators. That finding helps explain why litigants complain that Appeals Council decisions are unpredictable, and the finding leads to a diagnosis: the Appeals Council is both too deferential, because it is unable to conduct holistic factual review, and not deferential enough, because it must reverse decisions for harmless errors.