Subjective Measures of Liberal Democracy

Abstract

Using democracy in empirical work requires accurate measurement. Yet, most policy and academic research presupposes the accuracy of available measures. This article explores judge-specific measurement errors in cross-national indicators of liberal democracy. The authors evaluate the magnitude of these errors in widely used measures of democracy and determine whether their results replicate during a 17-year period (1972 to 1988). Then, they examine the nature of these systematic errors, hypothesizing that three different processes—(a) the information available for rating, (b) the judges' processing of this information, and (c) the method by which a judge's processing decisions are translated into a rating—could create error. The authors find that for the 17-year period from 1972 to 1988, there is unambiguous evidence of judge-specific measurement errors, which are related to traits of the countries. In the conclusion, the authors discuss the implications for democracy research and for other subjective measures.

Economic and Social Factors Driving the Third Wave of Democratization

“…The estimated likelihood that the country at the 25th percentile of education in 1975 will remain autocratic is 72.1%, while the chance that the country at the 25th percentile of education will not experience a transition is just 13%. 18 Thus the model predicts that the most likely outcome for countries with high levels of schooling was to experience a full democratization. The most likely outcome for countries with low levels of education was to stay autocratic, and the most likely scenario for countries with intermediate level of schooling was to implement moderate partial democratic reforms.…”

Measuring civil liberty: an assessment of standards-based data sets

“…second, one cannot be certain that even very high correlations lead to similar results as shown by a number of studies that have examined the impact of using different measures of democracy (Bollen and paxton 2000;casper and tufis 2003;Hadenius and teorell 2005). third, once again focusing on the five measures with the broadest scope, i have explored the number of outspoken differences between the indices most highly correlated.…”

“…Whereas the most prominent democracy indices have been scrutinized intensively (e.g. Bollen & paxton, 2000;munck & verkuilen, 2002;Lauth, 2004), this does not apply to civil liberty measures. although recent attempts to close this gap have emerged (e.g.…”

The Varieties of Democracy Core Civil Society Index

“…Thus, we utilize Bayesian item response theory models to aggregate their diverse ratings. These models are useful because they incorporate the information encoded in the variation in raters' perceptions, and in reliability levels across and within coders into the estimation process (Bollen andPaxton 2000, Jackman 2004 (King and Wand 2006). If these coders provide ratings for disjointed units, model estimates will be problematic since estimates for different countries will be based on different scales.…”