Identification of Preferences in Forced-Choice Conjoint Experiments: Reassessing the Quantity of Interest

Ganter, Flavien

doi:10.31235/osf.io/e638u

Cited by 7 publications

(12 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, I present a method for estimating ACPs. Proofs of consistency of the estimators are available, along with Monte-Carlo simulation evidence, in the supplemental information D. An example of implementation of this method is available online as an R function (Ganter 2021).…”

Section: Estimation and Inference Of The Acpmentioning

confidence: 99%

Identification of Preferences in Forced-Choice Conjoint Experiments: Reassessing the Quantity of Interest

Ganter¹

2021

Polit. Anal.

Self Cite

View full text Add to dashboard Cite

Forced-choice conjoint experiments have become a standard component of the experimental toolbox in political science and sociology. Yet the literature has largely overlooked the fact that conjoint experiments can be used for two distinct purposes: to uncover respondents’ multidimensional preferences, and to estimate the causal effects of some attributes on a profile’s selection probability in a multidimensional choice setting. This paper makes the argument that this distinction is both analytically and practically relevant, because the quantity of interest is contingent on the purpose of the study. The vast majority of social scientists relying on conjoint analyses, including most scholars interested in studying preferences, have adopted the average marginal component effect (AMCE) as their main quantity of interest. The paper shows that the AMCE is neither conceptually nor practically suited to explore respondents’ preferences. Not only is it essentially a causal quantity conceptually at odds with the goal of describing patterns of preferences, but it also does generally not identify preferences, mixing them with compositional effects unrelated to preferences. This paper proposes a novel estimand—the average component preference—designed to explore patterns of preferences, and it presents a method for estimating it.

show abstract

Section: Estimation and Inference Of The Acpmentioning

confidence: 99%

Identification of Preferences in Forced-Choice Conjoint Experiments: Reassessing the Quantity of Interest

Ganter¹

2021

Polit. Anal.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Though they tackle important limitations of the conjoint design, they often lack insight on minimal sample size requirements. For example, Ganter (2019) proves that the AMCE is not well suited to answer preferencerelated questions since it is influenced not only by the preference of the respondent on a given attribute but also by how the attributes are distributed in the population of interest (see also Abramson, Koçak, Statistical Power in Conjoint Experiments and Magazinnik 2019). As such, Ganter proposes a new estimator-the average component preference (ACP)-that decomposes the AMCE into an effect of preference and effect of composition (how are attributes distributed).…”

Section: Discussionmentioning

confidence: 99%

Subjects, Trials, and Levels: Statistical Power in Conjoint Experiments

Stefanelli¹,

Lukac²

2020

Preprint

View full text Add to dashboard Cite

Conjoint analysis is an experimental technique that has become quite popular to understand people's decisions in multi-dimensional decision-making processes. Despite the importance of power analysis for experimental techniques, current literature has largely disregarded statistical power considerations when designing conjoint experiments. The main goal of this article is to provide researchers and practitioners with a practical tool to calculate the statistical power of conjoint experiments. To this end, we first conducted an extensive literature review to understand how conjoint experiments are designed and gauge the plausible effect sizes discovered in the literature. Second, we formulate a data generating model that is sufficiently flexible to accommodate a wide range of conjoint designs and hypothesized effects. Third, we present the results of an extensive series of simulation experiments based on the previously formulated data generation process. Our results show that---even with relatively large sample size and the number of trials---conjoint experiments are not suited to draw inferences for experiments with large numbers of experimental conditions and relatively small effect sizes. Specifically, Type S and Type M errors are especially pronounced for experimental designs with relatively small effective sample sizes (< 3000) or a high number of levels (> 15) that find small but statistically significant effects (< 0.03). The proposed online tool based on the simulation results can be used by researchers to perform power analysis of their designs and hence achieve adequate design for future conjoint experiments.

show abstract

“…16 The counter-factual comparison refers to level-changes within a profile (Bansak 2020b). As AMCEs are estimated on the same scale, they are substantively comparable, yet bounded by the number of levels (Leeper, Hobolt and Tilley 2020;Ganter 2020). Therefore, inter-attribute comparisons have to be conducted with some care.…”

Section: Main Effectsmentioning

confidence: 99%

“…AMCEs (and MMs) do not capture (average) preferences over outcomes in the paired choice sense -when comparing two alternatives that differ on a level of an attribute (i.e. when comparing contrasting profiles) -which may be problematic if profile compositional effects are not of interest (Ganter 2020). Thus, Ganter (2020) and, similarly, Abramson et al (2020) both suggest a different quantity calculated only on contrasting profiles.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Deontological and Consequentialist Preferences Towards Arms Exports -- A Comparative Conjoint Experiment in France and Germany

Rudolph¹,

Freitag²,

Thurner³

2021

Preprint

View full text Add to dashboard Cite

Despite fierce politicization and heated public debates in arms-exporting democracies, systematic research on mass public preferences on arms trade is lacking. Combining political economy models of arms trade with the literatures on trade preferences and foreign policy attitudes, we argue that citizens trade off economic incentives, strategic interests and moral considerations when assessing arms trade and that deeply rooted `strategic cultures’ lead to differences in citizen preferences between countries. To derive the implicit weighting of different features of arms trade, we draw on population-representative conjoint survey experiments (N=6,617), fielded in November/December 2020 in two of the global top-5 exporting countries of major arms: Germany and France. We find that both country populations show structured preferences towards arms exports which predominantly center around their moral repercussions. However, German respondents place more weight on moral consequences and, compared to French respondents, a larger share is in fundamental opposition. We conclude that these diverging preferences potentially conflict with plans of a common European defense and security policy.

show abstract

Identification of Preferences in Forced-Choice Conjoint Experiments: Reassessing the Quantity of Interest

Cited by 7 publications

References 12 publications

Identification of Preferences in Forced-Choice Conjoint Experiments: Reassessing the Quantity of Interest

Identification of Preferences in Forced-Choice Conjoint Experiments: Reassessing the Quantity of Interest

Subjects, Trials, and Levels: Statistical Power in Conjoint Experiments

Deontological and Consequentialist Preferences Towards Arms Exports -- A Comparative Conjoint Experiment in France and Germany

Contact Info

Product

Resources

About