Determining subgroup difference importance with complex survey designs: An application of weighted dominance analysis

Abstract

Objective: Determining which subgroups show the most substantial differences on a measure is a common use of surveys. How to accurately and fairly determine which subgrouping is most important has not been addressed adequately in the literature. I show how dominance analysis is a useful way to identify the most important subgroup differences. Because surveys commonly employ complex sampling designs, I also provide practical guidelines for determining subgroup relative importance from complex survey data. Methods: The advantages of dominance analysis over alternative analysis procedures for determining importance are discussed using an empirical example from the political party affiliation question in the General Social Survey. Additionally, a Monte Carlo simulation was conducted to examine the accuracy of dominance analysis with complex sampling accounting for sample weights, strata, both, and neither compared to with known population values. Results: Dominance analysis clearly identifies the urbanicity subgrouping as having the most important differences on political party affiliation. Results also show the use of survey weights can have non-trivial effects on subgroup rank ordering. The simulation shows that weighed dominance statistics were more accurate than unweighted statistics. Stratified analyses had no effect on relative importance statistics. Conclusions: Dominance analysis is a useful way to identify key subgroup differences on survey measures. Survey weights are necessary to use for relative importance analysis, when available, in order to obtain an accurate representation of the rank order and magnitude of differences between subgroup indicators on a survey measure. The article concludes by outlining situations where dominance analysis is recommended. introduction Surveys are used to measure the prevalence of a behavior or attitude in a population but also to evaluate subgroup differences in prevalence. In fact, the US Census Bureau's website provides most measures collected cross-classified by demographic subgroup. Polls are used similarly, commonly testing for subgroup differences (e.g., Tables A1-A3 in Ramirez 2013). Survey research also seeks to determine which subgroup prevalence differences Institution: Fors Marsh Group LLC * Determining subgroup difference importance with complex survey designs: An application of weighted dominance analysis

Diverging ideas of health? Comparing the basis of health ratings across gender, age, and country

“…In order to assess the health domains’ contributions to R 2 , we conducted dominance analyses with the Stata-module domin ( Luchman, 2013 ). This approach compares R 2 for all possible subsets of variables or variable-sets in order to determine the variance explained by them or, in other words, their contribution to overall R 2 ( Budescu, 1993 ; Luchman, 2014 , 2015 ). If there are differences in the importance of dimensions between genders, age groups, or respondents from different countries, they are reflected in these analyses accordingly.…”

Cost-related medication non-adherence among U.S. adults with diabetes

“…Unlike other methods of computing relative importance, dominance analysis provides meaningful numerical measures of relative contribution of each variable in explaining CRN [29] that can be standardized to the amount of CRN explained in the percentage scale. Luchman (2015) showed that dominance analysis can be applied to complex surveys by using the survey weights as probability weights in logistic regression models [30]. …”

Correlates of depressive disorders and quality of life among older adults in six low and middle-income countries: WHO country-wide study on global ageing and adult health

“…To estimate the effect of a particular domain on DD, we adjust for the remaining eight domains. We assumed that the variables selected from DSLASSOPM are the only known independent predictors of DD and therefore applied weighted dominance regression analysis [22] to determine the relative importance of these predictors. Dominance analysis is an ensemble method that ranks the predictors in terms of importance by aggregating results across multiple models.…”