Estimating the Associations between SNAP and Food Insecurity, Obesity, and Food Purchases with Imperfect Administrative Measures of Participation

Abstract

Administrative data are considered the “gold standard” when measuring program participation, but little evidence exists on their potential problems or implications for econometric estimates. We explore these issues using the FoodAPS, a unique data set containing two different administrative measures of Supplemental Nutrition Assistance Program (SNAP) participation and a survey‐based measure. We document substantial ambiguity in the two administrative measures and show that they disagree with each other almost as often as they disagree with self‐reported participation. Estimated participation and misreporting rates can be meaningfully sensitive to choices made to resolve this ambiguity and disagreement. We explore sensitivity in regression estimates of the associations between SNAP and food insecurity, obesity, and the healthy eating index. The signs are unchanged across the three measures, and the estimates are mostly not statistically different from each other. However, there are some meaningful differences in the magnitudes and levels of statistical significance of the estimates.

Misreporting of Government Transfers: How Important Are Survey Design and Geography?

“…In addition, studying geographic variation in survey error requires the accuracy of the linked administrative variable not to vary between states. Courtemanche et al (2018) show that variation between states also stems from differences in data quality or linkage rather than from differences in reporting only. For the sample we use, we have no evidence of systematic differences in the accuracy of the linked data between states.…”

“…Consequently, we only analyze data from the 13 states that provided the most accurate and homogeneous data to minimize confounding survey error with variation in administrative data and linkage quality. 4 Courtemanche et al (2018) and Kang and Moffitt (2018) use the data from all states and examine their accuracy and usefulness further. They also find differences in the linked administrative variable between the 13 states that provided caseload and ALERT data with unique IDs and the remaining states.…”

The short‐term effects of the earned income tax credit on health care expenditures among US adults

“…MEPS queries subjects about whether they received an EITC refund but not the amount, and numerous prior studies have shown that self-reported receipt of EITC and other welfare benefits is neither sensitive nor specific and may bias results. [37][38][39][40] We therefore assumed that all individuals received the refund for which they were eligible, which is analogous to an intentto-treat approach in a randomized controlled trial. Prior work has shown that approximately 80 percent of eligible individuals during this time period received their refunds, 41 which means that this technique suffers from a degree of misclassification.…”

Investigating Treatment Effects of Participating Jointly in SNAP and WIC when the Treatment Is Validated Only for SNAP

“…Using ALERT, a household is considered to be a true current SNAP participant if the matching procedure indicated a disbursement during a 36-day window before the survey date. Courtemanche, Denteh, and Tchernis (2018), among other researchers, point out limitations of SNAP administrative records employed in FoodAPS, including missing data and varying degree of data quality across states. Also, the two administrative data sources sometimes disagree, perhaps in part due to underlying timing discrepancies.…”