Inference for Ranks with Applications to Mobility across Neighbourhoods and Academic Achievement across Countries

Abstract

It is often desired to rank different populations according to the value of some feature of each population. For example, it may be desired to rank neighborhoods according to some measure of intergenerational mobility or countries according to some measure of academic achievement. These rankings are invariably computed using estimates rather than the true values of these features. As a result, there may be considerable uncertainty concerning the rank of each population. In this paper, we consider the problem of accounting for such uncertainty by constructing confidence sets for the rank of each population. We consider both the problem of constructing marginal confidence sets for the rank of a particular population as well as simultaneous confidence sets for the ranks of all populations. We show how to construct such confidence sets under weak assumptions. An important feature of all of our constructions is that they remain computationally feasible even when the number of populations is very large. We apply our theoretical results to re-examine the rankings of both neighborhoods in the United States in terms of intergenerational mobility and developed countries in terms of academic achievement. The conclusions about which countries do best and worst at reading, math, and science are fairly robust to accounting for uncertainty. The confidence sets for the ranking of the 50 most populous commuting zones by measures of mobility are also found to be small. These rankings, however, become much less informative if one includes all commuting zones, if one considers neighborhoods at a more granular level (counties, Census tracts), or if one uses movers across areas to address concerns about selection.

A Critical Assessment of the National Expert Survey Data of the Global Entrepreneurship Monitor

“…Such a ranking is an important part of the yearly GEM reports but ignores the imprecision of the mean scores. Recently, Mogstad et al (2022) developed a methodology to derive confidence sets for rankings that account for the uncertainty surrounding the estimated mean scores. The goal of this methodology is to construct confidence sets that contain the ranks of all countries with a probability approximately not less than some pre-specified level (e.g., 95%).…”

“…As such, year-to-year changes in the ranking of a particular country based on NES data cannot be interpreted as structural changes. 11 Therefore, if GEM wants to maintain country rankings in their future reports, it is advised to adopt Mogstad et al (2022)'s methodology or similar methodologies to report rankings with an adequate level of precision. The confidence sets may reduce the plausible ''arms race'' based on differences in rankings that may not be statistically different.…”

“…9 Then, given the simultaneous confidence sets, the ranking is determined by how many confidence sets for a particular country lie above and below zero (because a difference of zero would indicate no difference in mean score). For a detailed description of the methodology, we refer readers to Mogstad et al (2022).…”

“…By applying a recently developed methodology (Mogstad et al, 2022) to the NES data, we further find that confidence sets for individual country ranks are very wide, and therefore, largely uninformative in separating countries by rankings. For example, in our 2018 analysis sample of 54 countries, the confidence set for the highest-ranking country (Indonesia) ranges from rank 1 to rank 22 for the composite National Entrepreneurship Context Index (NECI) while it ranges from rank 44 to rank 54 for the lowest-ranking country (Mozambique).…”

Joint Confidence Region Approach to Ranking Hotspot Locations Considering Uncertainty in Expected Risk Estimates

“…Mogstad et al ( 24 ) also conducted a similar analysis to that of Klein et al ( 22 ) to construct confidence sets for the joint ranking of a population. The only difference in their research is that they exploited joint confidence sets for the differences in the estimator value of certain population pairs instead of constructing simultaneous confidence sets for all population entities.…”