Computational Identification of Media Frames: Strengths, Weaknesses, and Opportunities

Abstract

With the availability of large volumes of electronic communications data and the increasing sophistication of computational techniques, the development of automated approaches for different kinds of framing analysis is an important goal of researchers. There is as yet no standard method for the 'unsupervised' inductive identification of frames based upon the content of articles. Three groups of core approaches underlie a wide range of work in this area, and we compare three techniques based on these approaches against each other and against manual human analysts. The three techniques are a k-means clustering algorithm together with a sophisticated natural language processing (NLP)-based feature selection process; evolutionary factor analysis (EFA), a factor analysis approach; and the structural topic model (STM). We use two datasets-one very focused and one extremely broad-as examples of the kind of frame analysis problems readers may wish to attempt. Even in a highly targeted dataset, we find some distance between the frames generated by computational analysis and those manually produced by our human analysts. The details of each method have a substantial impact on frame quality and interpretation. We find that the STM approach is the most effective for our narrow-scope dataset, but that it returns definite topics, and not frames, when working on our very broad dataset. We also show that we can get results emphasizing different parts of the framing problem by combining parts of different methods as a multi-stage process, rather than viewing available methods as simple plug-and-play models.

The discursive construction of digitalization: a comparative analysis of national discourses on the digital future of work

“…We prioritized quality over quantity (cf. Nicholls and Culpepper, 2020), selecting only articles that contained an explicit reference to the gig economy, workplace automation and robotization (including artificial intelligence), or the Fourth Industrial Revolution or Industry 4.0 in their headline or lead paragraph (for details, see online Appendix A.1). This corpus represents what Schmidt (2008) calls the communicative discourse between policymakers and the public and is broadly reflective of the overall public discourse on digitalization in a country.…”

“…because it tells us something about dominant framings), so we can use bag-of-words approaches like topic models (cf. Nicholls and Culpepper, 2020). If, however, we assumed words derive their meaning from 'the company they keep' and are interested in how their meaning changes over time or across contexts, word embeddings would be a more fitting approach.…”

A Mixed Methods Study of Public Perception of Social Distancing: Integrating Qualitative and Computational Analyses for Text Data

“…Among the computational methods that identify topics, we chose the structural topic model to analyze the participants’ open-ended responses (e.g., Chen et al, 2020; Roberts et al, 2014). This approach is found to be the most effective among various unsupervised methods in identifying topics and frames against manual coding, especially for narrow-scope data sets (Nicholls & Culpepper, 2021), and it provides more accurate estimation than other automated approaches, such as the Latent Dirichlet Allocation (Roberts et al, 2014).…”

Der „Computational Turn“: ein „interdisziplinärer Turn“? Ein systematischer Überblick zur Nutzung der automatisierten Inhaltsanalyse in der Journalismusforschung

“…Kritisch wird auch diskutiert, ob komplexere Darstellungsschemata, z. B. das Framing von spezifischen Inhalten, automatisiert zu erheben sind (Nicholls & Culpepper, 2021). Die Setzung von Inhalten durch spezifische Akteu-…”