Despite the popularity of unsupervised techniques for political science text-as-data research, the importance and implications of preprocessing decisions in this domain have received scant systematic attention. Yet, as we show, such decisions have profound effects on the results of real models for real data. We argue that substantive theory is typically too vague to be of use for feature selection, and that the supervised literature is not necessarily a helpful source of advice. To aid researchers working in unsupervised settings, we introduce a statistical procedure and software that examines the sensitivity of findings under alternate preprocessing regimes. This approach complements a researcher's substantive understanding of a problem by providing a characterization of the variability changes in preprocessing choices may induce when analyzing a particular dataset. In making scholars aware of the degree to which their results are likely to be sensitive to their preprocessing decisions, it aids replication efforts.preText software available: github.com/matthewjdenny/preText Word count: 10461 (excluding online appendices)

No abstract

Poole's (2000, Non-parametric unfolding of binary choice data. Political Analysis 8:211–37) nonparametric Optimal Classification procedure for binary data produces misleading rank orderings when applied to the modern House of Commons. With simulations and qualitative evidence, we show that the problem arises from the government-versus-opposition nature of British (Westminster) parliamentary politics and the strategic voting that is entailed therein. We suggest that political scientists think seriously about strategic voting in legislatures when interpreting results from such techniques.

Cohesive government-vs-opposition voting is a robust empirical regularity in Westminster democracies. Using new data from the modern Scottish parliament we show that this pattern cannot be explained by similarity of preferences within or between the government and opposition ranks. We look at differences in the way that parties operate in Westminster and Holyrood and use roll call records show that the observed behavior is unlikely to be determined by preferences on any underlying issue dimension. Using a simple variant of the agenda-setting model-in which MPs can commit to their voting strategies-we show that the procedural rules for reaching collective decisions in Westminster systems can explain this phenomenon: in the equilibrium, on some bills, members of the opposition vote against the government irrespective of the proposal that is made. Such strategic opposition can reinforce government cohesiveness and have a moderating effect on policy outcomes. We introduce

Native Americans are unique among domestic actors in that their relations with the U.S. government involve treaty making, with almost 600 such documents signed between the Revolutionary War and the turn of the twentieth century. We investigate the effect of constitutional changes to the treating process in 1871, by which Congress stripped the president of his ability to negotiate directly with tribes. We construct a comprehensive new data set by digitizing all of the treaties for systematic textual analysis. Employing scaling techniques validated with word-use information, we show that a single dimension characterizes the treaties as more or less "harsh" in land and resource cession terms. We find that specific institutional changes to treaty-making mechanisms had little effect on agreement outcomes. Rather, it is the relative bargaining power of the United States economically and militarily that contributes to worsening terms for Indians over the nineteenth century.

Measuring the polarization of legislators and parties is a key step in understanding how politics develops over time. But in parliamentary systems—where ideological positions estimated from roll calls may not be informative—producing valid estimates is extremely challenging. We suggest a new measurement strategy that makes innovative use of the “accuracy” of machine classifiers, i.e., the number of correct predictions made as a proportion of all predictions. In our case, the “labels” are the party identifications of the members of parliament, predicted from their speeches along with some information on debate subjects. Intuitively, when the learner is able to discriminate members in the two main Westminster parties well, we claim we are in a period of “high” polarization. By contrast, when the classifier has low accuracy—and makes a relatively large number of mistakes in terms of allocating members to parties based on the data—we argue parliament is in an era of “low” polarization. This approach is fast and substantively valid, and we demonstrate its merits with simulations, and by comparing the estimates from 78 years of House of Commons speeches with qualitative and quantitative historical accounts of the same. As a headline finding, we note that contemporary British politics is approximately as polarized as it was in the mid-1960s—that is, in the middle of the “postwar consensus”. More broadly, we show that the technical performance of supervised learning algorithms can be directly informative about substantive matters in social science.

No abstract

No abstract