Collaborating with the Machines: A Hybrid Method for Classifying Policy Documents

Abstract

Governments produce vast and growing quantities of freely available text: laws, rules, budgets, press releases, and so forth. This information flood is facilitating important, growing research programs in policy and public administration. However, tightening research budgets and the information's vast scale forces political science and public policy to aspire to do more with less. Meeting this challenge means applied researchers must innovate. This article makes two contributions for practical text coding—the process of sorting government text into researcher‐defined coding schemes. First, we propose a method of combining human coding with automated computer classification for large data sets. Second, we present a well‐known algorithm for automated text classification, the Naïve Bayes classifier, and provide software for working with it. We argue and provide evidence that this method can help applied researchers using human coders to get more from their research budgets, and we demonstrate the method using classical examples from the study of policy agendas.

Big Data and AI – A transformational shift for government: So, what next for research?

“…It supports public managers in aggregating and analysing citizens' policy preferences and needs in order to better understand which incentives will work and under what circumstances (Clarke and Margetts, 2014). Moreover, compared to traditional data, Big Data allows for a variety of sources to be researched and analysed to understand the policy landscape (Loftis and Mortensen, 2018; Nowlin, 2016), identify problems, and conceptualise solutions more accurately and in greater detail (Williams, 2014a). In policy formulation, Big Data helps to design a policy that closely matches the preferences (Stritch et al., 2017; Taeihagh, 2017), in addition to developing different scenarios and accurately predicting their possible outcomes (Cook, 2014).…”

The Bureaucracy and the Policy Agenda

“…SeeLoftis and Mortensen (2017) for a description of the coding strategy, including intercoderreliability tests.…”

Using Multi-Factor Analysis to Predict Urban Flood Depth Based on Naive Bayes

“…A variety of methods can be used to predict urban flood depth, but complex structures and effective predictions require time and computational resources. Even small processes can put massive demands on the computer [23]. Naive Bayes (NB) has shown exceptional speed and accuracy with large data sets [24].…”