Spatial Variation in the Quality of American Community Survey Estimates

Abstract

Social science research, public and private sector decisions, and allocations of federal resources often rely on data from the American Community Survey (ACS). However, this critical data source has high uncertainty in some of its most frequently used estimates. Using 2006-2010 ACS median household income estimates at the census tract scale as a test case, we explore spatial and nonspatial patterns in ACS estimate quality. We find that spatial patterns of uncertainty in the northern United States differ from those in the southern United States, and they are also different in suburbs than in urban cores. In both cases, uncertainty is lower in the former than the latter. In addition, uncertainty is higher in areas with lower incomes. We use a series of multivariate spatial regression models to describe the patterns of association between uncertainty in estimates and economic, demographic, and geographic factors, controlling for the number of responses. We find that these demographic and geographic patterns in estimate quality persist even after we account for the number of responses. Our results indicate that data quality varies across places, making cross-sectional analysis both within and across regions less reliable. Finally, we present advice for data users and potential solutions to the challenges identified.

Spatial Autocorrelation Statistics of Areal Prevalence Rates under High Uncertainty in Denominator Data

“…A single census tract located in the northeastern part of the county has "High" reliability. This is largely consistent with the spatial pattern of data uncertainty of household median income from ACS, which tends to exhibit a spatial pattern where data uncertainties increase towards the urban center (Folch et al 2016).…”

“…The authors found that poorer and inner city neighborhoods tend to have higher coefficients of variation in median income estimates than richer and outer neighborhoods. This conclusion was found by (Folch et al ) to be valid for the Unites States at large as well. Case‐based community survey has also confirmed that ACS estimates tend to be less accurate in poorer neighborhoods (Bazuin and Fraser ).…”

“…Case-based community survey has also confirmed that ACS estimates tend to be less accurate in poorer neighborhoods (Bazuin and Fraser 2013). Therefore, practical uses of ACS should be done with critical consideration of data quality (Macdonald 2006;Spielman, Folch, and Nagle 2014;Folch et al 2016). Moreover, Sun and Wong (2010) showed that, due to the large margins of error of ACS estimates, calculating year-to-year difference of ACS estimates may generate very biased outcomes.…”

Fast Food Data: Where User‐Generated Content Works and Where It Does Not

“… It should be noted that even federal sources such as the American Community Survey are not immune to data quality issues (Folch et al ).…”

“…7 All computations were done using the Jellyfish python library (Turk and Stephens 2016) 8 The percentages presented here slightly overstate the discrepancy because the CBP data considers the entire county and therefore includes small outlying communities we removed from the MAG and Yelp data sets. 9 It should be noted that even federal sources such as the American Community Survey are not immune to data quality issues (Folch et al 2016).…”