Inference for Ranks with Applications to Mobility Across Neighborhoods and Academic Achievement Across Countries

Abstract

It is often desired to rank different populations according to the value of some feature of each population. For example, it may be desired to rank neighborhoods according to some measure of intergenerational mobility or countries according to some measure of academic achievement. These rankings are invariably computed using estimates rather than the true values of these features. As a result, there may be considerable uncertainty concerning the rank of each population. In this paper, we consider the problem of accounting for such uncertainty by constructing confidence sets for the rank of each population. We consider both the problem of constructing marginal confidence sets for the rank of a particular population as well as simultaneous confidence sets for the ranks of all populations. We show how to construct such confidence sets under weak assumptions. An important feature of all of our constructions is that they remain computationally feasible even when the number of populations is very large. We apply our theoretical results to re-examine the rankings of both neighborhoods in the United States in terms of intergenerational mobility and developed countries in terms of academic achievement. The conclusions about which countries do best and worst at reading, math, and science are fairly robust to accounting for uncertainty. By comparison, several celebrated findings about intergenerational mobility in the United states are not robust to taking uncertainty into account.

Early Exposure to County Income Mobility and Adult Individual Health in the United States

“…In contrast, the NLSY97 and PSID samples cover only 55% and 39% of the counties, respectively. Although we do not know what would happen to previous research results if only the counties of NLSY97 and PSID samples were considered, or better, if the measurement error were considered in the analysis, our findings and the work by Mogstad et al (2020) suggest that measurement error might exaggerate estimates. Future research will be needed to assess the consequences of measurement error thoroughly, provided the standard errors of income mobility estimates are available.…”

“…First, our results might be affected by measurement error. As recently Mogstad et al (2020) have pointed out, income mobility measures and rankings computed by Chetty et al (2014)'s and colleagues are estimates rather than true values, so they might carry considerable uncertainty as population size varies considerably across counties. Figure S1 in the Methodological Supplement provides some evidence on how measurement error might affect our results.…”

Inference on winners

“…Conventional inference procedures for these problems fail for similar reasons that give rise to a winner's curse. Recent work by Mogstad et al (2020) overcomes this inference failure and studies, among other settings, inference on ranks in neighborhood targeting, as in our second application.…”

“…It is useful to compare our results with those ofMogstad et al (2020), who study the problem of inference on ranks and consider the Opportunity Atlas data for Seattle as an example. They show that if one forms simultaneous confidence sets for individual tracts, one can say very little about which tracts are best.…”

Intergenerational Mobility in Africa

“…Many studies focus on (one minus) the intergenerational coefficient obtained from a regression of children on parental schooling (e.g., Hertz et al (2008)); others work with rank‐rank correlation coefficients and intergenerational rank movements (e.g., Asher, Novosad, and Rafkin (2020), Geng (2018), Chetty et al (2014)). While rank‐based measures isolate the relative movement of children in the distribution compared to the older generation from the overall increase, they may be sensitive to measurement error (see Mogstad, Romano, Shaikh, and Wilhelm (2020)). Other studies (e.g., Card, Domnisoru, and Taylor (2018), Davis and Mazumder (2020) and Chetty et al (2017)) focused, as we do, on absolute transition likelihoods.…”