Using Small Area Estimation to Produce Official Statistics

Abstract

The USDA National Agricultural Statistics Service (NASS) and other federal statistical agencies have used probability-based surveys as the foundation for official statistics for over half a century. Non-survey data that can be used to improve the accuracy and precision of estimates such as administrative, remotely sensed, and retail data have become increasingly available. Both frequentist and Bayesian models are used to combine survey and non-survey data in a principled manner. NASS has recently adopted Bayesian subarea models for three of its national programs: farm labor, crop county estimates, and cash rent county estimates. Each program provides valuable estimates at multiple scales of geography. For each program, technical challenges had to be met and a strenuous review completed before models could be adopted as the foundation for official statistics. Moving models out of the research phase into production required major changes in the production process and a cultural shift. With the implemented models, NASS now has measures of uncertainty, transparency, and reproducibility of its official statistics.

Smoothing County-Level Sampling Variances to Improve Small Area Models’ Outputs

“…Appendix A. Mathematical Feasures for Model (2) It is pertinent to give a mathematical explanation of the features of the model in Equation (2). For this discussion, we assume that β and δ 2 are fixed but unknown.…”

“…Note that φ * i = δ2 /( θi − x i β) 2 are not really MLEs because the numerator may be zero and φ * i may not be in the open interval (0, 1). Therefore, when letting λi = δ2 /( δ2 + σ2 i ), we have…”

“…Yield is defined as the ratio of total production to the harvested acreage. Starting in 2020, several HB subarea-level (small area) models have been implemented as extensions of the FH model to improve the precision of the estimates at the county level (see [2][3][4][5][6]). The sampling variances of the yield estimates are produced using a second-order Taylor series approximation and, due to various reasons (e.g., sparseness in data), could result in zero, very small or very large estimated variances for several counties.…”

A scaled kernel density estimation prior for dynamic borrowing of historical information with application to clinical trial design

“…However, determining how to most effectively borrow historical information in a Bayesian setting is an open question, as incorrectly borrowing (ie, when the historical and current data are in disagreement) can result in inaccurate statistical inference, while failing to utilize supporting information is inefficient. To this end, several methods that allow for data‐driven borrowing of historical information to improve statistical inference in the current data analysis have been introduced (eg, References 1‐3) and are becoming increasingly used in practice across many different scientific disciplines (eg, References 4‐10). We detail several of these major methodological categories in Section 3.…”

LEAP: The latent exchangeability prior for borrowing information from historical data

“…Such prior information, such as historical data, is naturally integrated in the Bayesian paradigm, where analysts may specify a prior distribution for what evidence they possess prior to the analysis of the data. Bayesian methods for incorporating historical data have been used or proposed in wide a variety of statistical applications including, but not limited to, epidemiology [Warasi et al, 2016], political science [Isakov and Kuriwaki, 2020], engineering [Lorencin and Pantoš, 2017], spatial applications [Louzada et al, 2021], small area estimation [Young and Chen, 2022], and psychology [König et al, 2021].…”