The case of restaurant hygiene grading occupies a central role in information disclosure scholarship. Comparing Los Angeles, which enacted grading in 1998, with California from 1995–1999, Jin and Leslie (2003) found that grading reduced foodborne illness hospitalizations by 20 percent. Expanding hospitalization data and collecting new data on mandatorily reported illnesses, we show that this finding does not hold up under improvements to the original data and methodology. The largest salmonella outbreak in state history hit Southern California before Los Angeles implemented grading. Placebo tests detect the same treatment effects for Southern California counties, none of which changed restaurant grading. (JEL D83, H75, I12, I18, L83, L88)
Evidence-based policy is limited by the perception that randomized controlled trials (RCTs) are expensive and infeasible. We argue that carefully tailored research design can overcome these challenges and enable more widespread randomized evaluations of policy implementation. We demonstrate how a stepped-wedge (randomized rollout) design that adapts synthetic control methods overcame substantial practical, administrative, political, and statistical constraints to evaluating King County’s new food safety rating system. The core RCT component of the evaluation came at little financial cost to the government, allowed the entire county to be treated, and resulted in no functional implementation delay. The case of restaurant sanitation grading has played a critical role in the scholarship on information disclosure, and our study provides the first evidence from a randomized trial of the causal effects of grading on health outcomes. We find that the grading system had no appreciable effects on foodborne illness, hospitalization, or food handling practices but that the system may have marginally increased public engagement by encouraging higher reporting.
We study a unique natural experiment, during which 5-10% of draft opinions by judges of the Board of Veterans Appeals (BVA) were randomly selected for "quality review" by a team of full-time staff attorneys for nearly 15 years. This performance program had the express goals of measuring accuracy and reducing reversal rates on appeal. In cases of legal error, the quality review team wrote memoranda to judges to permit correction before opinions were issued. We use rich internal administrative data on nearly 600,000 cases from 2002-2016 to provide the first rigorous study of this review process. With precise estimates, we show that the program had no appreciable effect on reducing appeals or reversals. Based on internal records, we demonstrate that this inefficacy is likely by design, as meeting the performance measure of "accuracy" was at cross-purposes with error correction. These findings inform longstanding questions of law, organization, and bureaucracy, including performance management, standards of review, and the institutional design of mass adjudication.
We analyze the results of a neighbor-to-neighbor, grassroots get-out-the-vote (GOTV) drive in Virginia, in which unpaid volunteers were encouraged to contact at least three nearby registered voters who were likely co-partisans yet relatively unlikely to vote in the 2017 state election. To measure the campaign’s effectiveness, we used a pairwise randomization design whereby each volunteer was assigned to one randomly selected member of the most geographically proximate pair of voters. Because some volunteers unexpectedly signed up to participate outside their home districts, we analyze the volunteers who adhered to the original hyper-local program design separately from those who did not. We find that the volunteers in the original program design drove a statistically significant 2.3% increase in turnout, which was concentrated in the first voter pair assigned to each volunteer. We discuss implications for the study and design of future GOTV efforts.
Broad stakeholder participation in regulatory policymaking via online commenting platforms has become the norm in many advanced democracies around the world. In recent years, a policy debate has emerged over the dangers posed to the process by fake comments that impersonate ordinary citizens. This paper helps to clarify the terms of this debate by evaluating a contentious and prominent case in the United States, the nearly 24 million comments from the Federal Communications Commission's 2017 Restoring Internet Freedom proceeding. Using a two‐step methodology that combines a computationally efficient search algorithm and a neural network language model, I show that regulators were able to cite much of the relevant information submitted in public comments by relying on longstanding methods of information gathering through interest groups, despite the fact that fake comments outnumbered others 3 to 1. The results suggest that fake comments did not impede regulators' ability to extract specialized information from the public consultation process, but may have distorted signals from mass comment campaigns about constituent mobilization.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.