Content Analysis, Automatic

Abstract

Automatic content analysis (ACA) is a technique for coding messages with the help of computer algorithms. Unlike computer-aided content analysis, ACA is defined as any method in which the actual coding decision, that is, assigning codes to documents or single textual or audiovisual elements, does not require human judgment and therefore is performed automatically. Since ACA relies on the computing capabilities of machines rather than human coders, it can be applied to very large documents. Moreover, automatic coding is highly reliable in that any analysis can be reproduced exactly given the same material and software and all errors are deterministic, that is, stemming from misspecification or program errors. Historical developmentThe history and development of ACA can be understood through three central themes: (i) the concept of content analysis and its computational implementation, (ii) the development of software for automatic data processing and analysis, and (iii) the provision and analysis of digital or machine-readable documents. The first phase of development of computer-assisted and automatic content analysis in the late 1950s was mainly characterized by experiments with the computer. For this, social scientists had to first learn to program the computer, and in most cases, this involved the support of the science departments. The first few experiments were almost exclusively limited to producing text statistics, that is, counting words, a technique that had been applied in many disciplines such as political science or literature since the 1920s (Holsti, 1969;Stone, Dunphy, Smith, & Ogilvie, 1966). Conceptually, these simple automatic analyses fell well behind the advancements made in conventional content analysis by Lasswell, Lerner, and de Sola Pool (1952) or Osgood (1959). However, early content analysts were increasingly troubled by the costs of manual coding and had high expectations from automatic approaches, the development of which was viewed as central to the success of the method (Stone, 1997). At this time, computer-assisted content analysis was a high-risk research domain, plagued with numerous problems. On the one hand, primitive computing hardware and software allowed only small amounts of text to be analyzed with a few variables (Iker & Harway, 1965). On the other hand, because of the dearth of machine-readable documents, all units of analysis had to be digitized in a complex and error-prone process on punch cards. Thus, in the early 1960s, ACA was neither cost-effective-half an

What’s the Tone? Easy Doesn’t Do It: Analyzing Performance and Agreement Between Off-the-Shelf Sentiment Analysis Tools

“…Consistent with general measurement theory, more reliable measurements should be yielded when multiple measures of the same construct are combined (Gonçalves, Araújo, Benevenuto, & Cha, 2013). However, this will only occur if they indeed measure the same underlying component (Scharkow, 2017), which is likely not always the case for off-the-shelf sentiment analysis tools (Soroka et al, 2015). As a combination of tone measurements potentially corrects for the weaknesses and unreliability of individual tools, we expect the following: H 1 : A combined approach will more strongly agree with human coding than the individual automatic measurements.…”

“…Fourth, several studies have applied dictionaries to automatically measure the tone of economic news (e.g., De Boef & Kellstedt, 2004;Shapiro, Sudhof, & Wilson, 2019;Soroka, 2012;Tetlock, 2007;Van Dalen, de Vreese, & Albaek, 2017). Their procedures were straightforward and relied on socalled "bag-of-words" approaches (see Scharkow, 2017): Counting the number of words in a text that are categorized as positive in a pre-established dictionary; counting the number of negative words in the same text; and eventually subtracting these from each other (see Young & Soroka, 2012, for detailed explanation)mostly without taking the syntactic structure of sentences into account.…”