Coping With Plenitude: A Computational Approach to Selecting the Right Algorithm

Abstract

Sociologists increasingly face choices among competing algorithms that represent reasonable approaches to the same task, with little guidance in choosing among them. We develop a strategy that uses simulated data to identify the conditions under which different methods perform well and applies what is learned from the simulations to predict which method will perform best on never-before-seen empirical data sets. We apply this strategy to a class of methods that group respondents to attitude surveys according to whether they share construals of a given domain. This allows us to identify the relative strengths and weaknesses of the methods we consider, including relational class analysis, correlational class analysis, and eight other such variants. Results support the “no free lunch” view that researchers should abandon the quest for one best algorithm in favor of matching algorithms to kinds of data for which each is most appropriate and provide direction on how to do so.

Different understandings of welfare benefits among the Dutch public: A focus group study

“…By analysing whether attitudes on, for instance, a specific welfare arrangement relate differently to other social and political attitudes among different segments of the population, one can uncover whether different meanings are ascribed to it (cf. Sotoudeh & DiMaggio, 2021, pp. 4–5).…”

Party, Race, and Neutrality: Investigating the Interdependence of Attitudes toward Social Groups

“…Our analyses are limited in a few ways. First, clustering methods like RCA can be sensitive to the set of variables that analysts choose to include as well as to the distance measures they use to compare respondents prior to clustering (Sotoudeh and DiMaggio 2021). In Part B of the online supplement, we examine the robustness of our results to a number of factors, including our choice of distance measure, the set of social groups included in our analysis, and sample selection.…”

“… 4. We used the method outlined by Sotoudeh and DiMaggio (2021) to ensure we chose the clustering measure best suited for our data. All the measures they provided were expected to perform exceptionally well on the data, especially relationality (with Louvain), correlation, eJaccard, and eDice.…”

“… 6. Louvain’s clustering measure is not hierarchical, so we could not consult gap statistics, such as that used by DiMaggio and Goldberg (2018), to assess model fit. Instead, following Boutyline and Vaisey (2017) and Sotoudeh and DiMaggio (2021), we used SEM to compare the three-group clustering solution identified by RCA to a baseline model in which all respondents are treated as having come from a single group. Results, explained in more detail in Part B of the online supplement, show the three-class RCA solution better fits the data. …”