In an earlier post in this series, two of us (Cortland and Tani) described how the notice-and-comment process has entered the arsenal of a range of groups and organizers, many seeking to challenge the policies of the current administration. We made the case by highlighting grassroots efforts to explain to the public what notice-and-comment is and how to participate in it. The effects of these efforts seem clear. Note the more than 1,800 comments on Kentucky’s 2016 request for a Medicaid Section 1115 waiver (seeking to impose a work requirement, among other changes). Or consider the 266,000+ comments to the Department of Homeland Security after the agency proposed a rule that would make an immigrant’s use (or likely use) of public benefits grounds for inadmissibility.

Here we discuss a type of organizing that has followed in the wake of mass commenting efforts, focused on analyzing the content of comments and ensuring their due consideration. Our central example comes from the Department of Education’s recent notice of a proposed rulemaking (NPRM) regarding the meaning of Title IX for complaints of sexual harassment (an umbrella legal term that includes sexual violence). Perhaps unsurprisingly, given the activism around the ED’s interpretation of Title IX in the Obama years, the agency recorded over 124,000 comments—and spurred the creative cataloging initiative we spotlight here. By analyzing comments themselves, participants in this initiative seek to support public engagement with administrative law and vindicate the democratic values at the heart of notice-and-comment mandates.

At first glance, this analysis may seem unnecessary. Under the Administrative Procedure Act, the notice-and-comment process requires agency representatives to examine the relevant data and show that they have adequately considered all significant aspects of the problem, on pain of judicial reversal. (For elaboration, see Donald Kochan’s useful 2018 “tutorial” on “the commenting power.”)

But judicial challenges take resources, and even well-resourced lawyers must work within one notable limit of the APA: it does not require agencies to make public their internal methods for analyzing comments, leaving many questions un-answered. For example, how do agencies treat “boilerplate” comments? The website where members of the public submit comments, Regulations.gov, states that “the comment process is not a vote” and that “one well supported comment is often more influential than a thousand form letters.” How much more influential? And how much should an agency discount form letters, especially when they come from commenters who may not have the time, education, or confidence to draft a polished submission from scratch? The public might like to know if 40,000 comments on a proposed rulemaking made a similar point or took the same position. But nothing in current law requires agencies to release these totals. (We are not alone in raising such questions. For a deeper exploration of the varying and sometimes opaque administrative practices around comment submissions, see this June 2019 Government Accountability Office Rulemaking Report.)

Regulations.gov also specifies, in small, footnoted print, that the number of comments it lists as filed, and the content of the comments it makes public, may not reflect actual submissions: Agencies are free to “redact, or withhold, certain submissions (or portions thereof) such as those containing private or proprietary information, inappropriate language, or duplicate/near duplicate examples of a mass-mail campaign.” This limited degree of transparency has disturbed some organizers, as have the anti-democratic aspects of these caveats. Concerns have been especially acute when the proposed administrative action will affect people whom the political process has historically disadvantaged and when the agency has a reputation for duplicity or corruption.

For all of these reasons, we are now seeing efforts at independent review—whereby non-government actors read publicly available comments and produce independent analyses.

This kind of review is difficult and time-intensive, especially when it aims to go beyond comments from regulated entities and well-resourced organizations. One of us (Cantalupo) learned this when attempting to review the more than 16,000 comments that the ED received after it invited public input “on regulations that may be appropriate for repeal, replacement, or modification.” Technically, this invitation was a required response to Executive Order 13777 (establishing a federal policy to “alleviate unnecessary regulatory burdens”), but people interested in Title IX perceived it as a referendum on the previous administration’s guidance regarding sexual harassment, which critics had assailed as excessive and intrusive. And, indeed, within the final weeks of the comment period, the ED announced its withdrawal of this guidance, having concluded that the Obama administration’s enforcement of Title IX was a “failed system” and “widely criticized.” A day spent reviewing comments gave Cantalupo reason to question these statements, as well as inspiration for a class project for her “Title IX Practicum” course. Her students read and catalogued all available comments, and the full research team (including a professional data manager) then analyzed the 12,035 submissions addressing Title IX. (The analysis demonstrated that commenters overwhelmingly supported the rescinded guidance).

A follow-on initiative is the Big Comment Catalog Project, which seeks to analyze responses to the 2019 Title IX NPRM. Adopting a crowd-sourcing approach, the Project invites volunteers to catalog small batches of comments. This entails reading assigned comments and categorizing them according to broad characteristics (for example, whether the commenter filed anonymously; whether the comment included a personal story or references to research; and which of the many major proposed policy changes—from the definition of sexual harassment to cross-examination in student disciplinary proceedings—the comment addressed). Project organizers identified the cataloging categories by consulting Title IX and administrative law experts, with the goal of creating an open, user-friendly archive of what the public said.

When completed, the comment catalog will be a vital tool. For researchers, it will offer searchability of what is likely the largest single repository of the general public’s views on sexual harassment in education. For parties interested in the current administration’s interpretation of Title IX, it will offer a more content-based and potentially egalitarian vehicle for determining which comments the ED ought to consider, because it will make more accessible the views of people who do not employ lobbyists or have well-developed media connections. And should government officials (or the media) misrepresent public opinion, the catalog will help the public hold them accountable.

To be sure, significant obstacles confront the Big Comment Catalog Project, and all projects like it. Most notably, this “people-powered” approach relies on unpaid volunteers, all of whom must have the time, capacity, and computer and internet access to do the cataloging work. For the Title IX NPRM comment catalog, completion before the ED finalizes its rule likely requires thousands of volunteers. (The Project’s research team has thus far not found an electronic method of completing its task that is not prohibitively expensive.) And even with timely catalog completion, these massive amounts of time and effort may, in the end, produce no appreciable effect on public policy. This could leave some volunteers feeling disillusioned with government and disinclined to continue this work. Efforts aimed at protecting and enhancing democracy might instead expose its faltering health.

On the other hand—and this is our hope—these comment-processing efforts might have salubrious consequences for democracy, even if they fail by some metrics. Participation can educate people about the views of their peers and the often-mysterious workings of the government’s administrative apparatuses. Participation also has the potential to infuse politics with the kind of righteous anger on display when Representative Maxine Waters famously “reclaimed her time”: the indignation of those who know that they have a right to be heard and have grown tired of being stonewalled.

Nancy Chi Cantalupo is an associate professor of law and author of And Even More of Us Are Brave: Intersectionality & Sexual Harassment of Women Students of Color. Matthew Cortland (@mattbc) is an attorney in Massachusetts, where his practice focuses on healthcare and disability rights. Karen Tani (@kmtani) is a Professor of Law at the University of California, Berkeley. This post is part of the Law and Political Economy Blog’s symposium on democratizing administrative law. You can find all the posts in the series here.