Estimating parties’ policy positions through voting advice applications: Some methodological considerations

Abstract

The past few years have seen the advent and proliferation of Voting Advice (or Aid) Applications (VAAs), which offer voting advice on the basis of calculating the ideological congruence between citizens and political actors. Although VAA data have often been used to test many empirical questions regarding voting behaviour and political participation, we know little about the approaches used by VAAs to estimate the positions of political parties. This article presents the most common aspects of the VAA approach and examines some methodological issues regarding the phrasing of statements, the format of response scales, the reliability of coding statements into response scales and the reliability and validity of scaling items into dimensions. The article argues that VAAs have a lot of potential but there is also much space for methodological improvements, and therefore concludes with some recommendations for designing VAAs.

Spatial maps in voting advice applications: The case for dynamic scale validation

“…Therefore, the need for empirical validation extends beyond smartvote. In line with this, scaling analyses by Gemenis (2013) and Louwerse and Otjes (2012) have shown that the EU Profiler's more standard scales were deficient (also see Germann and Mendez, 2013). Hence, given the growing number of VAA users worldwide, the upshot of scale validation, and dynamic scale validation in particular, is potentially very large indeed in terms of more valid and reliable spatial matches.…”

“…6 Smartvote users had the option of filling in a shorter version of the questionnaire. 7 There are also substantive reasons against including spending items (Gemenis, 2013). Meanwhile, the differences between the original scales (with spending items) and our shorter scales (without spending items) are marginal (ρ c = 0.98 for both).…”

“…Recent evaluations suggest that VAA spatial maps did not always meet basic measurement criteria (Louwerse and Otjes, 2012;Gemenis, 2013). Scrutinizing the EU Profiler's (the first pan-European VAA designed for the 2009 European elections) two-dimensional map, both Gemenis and Louwerse and Otjes concluded that its left-right scale was deficient in terms of unidimensionality.…”

“…However, matching voters to political actors is simple only in the abstract. Recent work highlights that various aspects related to VAA design affect the quality of the ideological congruence estimates, including item wording (Gemenis, 2013), item selection (Walgrave et al, 2009), the matching algorithm (Mendez, 2012) and the rating scale format (Baka et al, 2012).…”

A perfect match? The impact of statement selection on voting advice applications' ability to match voters and parties

“…If statements severely advantage or disadvantage some parties, VAA builders can at least knowingly include or exclude them. Seeing the large differences in output caused by statements selections that differ only by one or two statements should sensitize VAA builders to the importance not only of the statement selection itself, but also of formulating statements according to standard practice in surveys-by avoiding double-barreledness, for example (Gemenis, 2013).…”