A Computational Approach to Qualitative Analysis in Large Textual Datasets

Evans, Michael S.

doi:10.1371/journal.pone.0087908

Cited by 64 publications

(60 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If possible, topics are then labeled manually (e.g. Newman & Block, 2006;Evans, 2014). A possible next step is the aggregation of the results of topic-inference for all documents of a period to analyze topics over time, e.g.…”

Section: Statistical Models Of Contents With Topic Modelingmentioning

confidence: 99%

See 1 more Smart Citation

Methods for user‐centered design and evaluation of text analysis tools in a digital history project

Heuwing

Mandl

Womser‐Hacker

2016

Proc. Assoc. Info. Sci. Tech.

View full text Add to dashboard Cite

This paper reports on the user centered, formative evaluation of tools and the validation of models for the analysis of historical textbooks in the context of the digital history project Children and their World. The goal of the project is to create methods for computer-supported, interactive analysis that can be applied to a large corpus of historical textbooks on history and geography (~5000 volumes). A first version of a tool for text analysis has been created based on a user centered design process, including a contextual study on current work practices of historians, participative design workshops, and the prioritization of requirements with the project stakeholders. In addition, several generations of text models used in these tools have been iteratively evaluated by the historians in the project. In the context of a cooperative validation study, researchers have used these tools to examine existing hypotheses from the field. The method enables the validation of text models regarding established knowledge, provides additional insights into the requirements for tools and visualizations, and helps to strengthen the expert users' trust in the tools. Based on the findings, this paper proposes four principles for the analysis of a corpus of historical texts. Moreover, the methods presented are discussed regarding the application of user centered design in the context of digital humanities projects.

show abstract

Section: Statistical Models Of Contents With Topic Modelingmentioning

confidence: 99%

“…In contrast, domain knowledge can also be stated beforehand as a set of hypotheses about the topics and their distributions in the corpus and over time (e.g. DiMaggio et al, 2013;Evans, 2014).…”

Section: Evaluation Of Topic Modelsmentioning

confidence: 99%

Methods for user‐centered design and evaluation of text analysis tools in a digital history project

Heuwing

Mandl

Womser‐Hacker

2016

Proc. Assoc. Info. Sci. Tech.

View full text Add to dashboard Cite

show abstract

“…The next step is to verify whether the results also have external validity. The strategy available to identify external validity is to observe the temporal pattern in which the topics occur and compare this pattern with real events that took place during the period analyzed (Newman et al, 2006;Evans, 2014).…”

Section: Results: What and When -International News In Brazilian Newsmentioning

confidence: 99%

“…However, if the model produces topics with words similar to "car," "paper," and "sky," it is not possible to find an interpretable label for that topic. Regarding external validity, for some researchers it is the ability of a topic model to capture events external to it (Newman et al, 2006;Evans, 2014). For example, if, when analyzing news stories that were written between September 8 and 12, 2001 in The New York Times, the topic model does not create a topic for terrorist attack, it means that the model has no external validity.…”

Section: Topic Modelingmentioning

confidence: 99%

Who, what and when: how media and politicians shape the Brazilian debate on foreign affairs

Hardt¹

View full text Add to dashboard Cite

What do politicians talk about when discussing foreign affairs? Are these topics different from the ones in the newspapers? Finally, can unsupervised methods be used to help us understand these problems? Answering these questions is of paramount importance to understanding the relationship between foreign policy and mass media. Based on this discussion, this research has three main objectives: (a) to verify whether unsupervised methods can be used to analyze documents on international issues; (b) to understand the issues that politicians talk about when dealing with foreign affairs; and (c) to understand when and with which periodicity the mass media publish news on certain international topics. To do so, I created two new corpora, one with news articles published in the international section of two major Brazilian newspapers; and a corpus with all speeches made within the two Committees on Foreign Affairs of the National Congress of Brazil. I ran a topic model using Latent Dirichlet Allocation (LDA) in both. The results of this topic model show that LDA can be used to distinguish different international issues that appear in both political discourse and the mass media in Brazil. Additionally, I found that the LDA model can be used to identify when some topics are debated and for how long. The findings also demonstrate that Brazilian politicians and Brazilian newspapers are neither isolated nor unstable in what regards international issues.

show abstract

“…The selection of an appropriate topic model involves a variety of tradeoffs and judgments by the human researcher (Evans, 2014); the selection of the model that is the best fit for the specific research question requires both qualitative and quantitative validation techniques . As noted above, one of the limitations of topic modeling is the requirement for the researcher to select the number of topics.…”

Section: Heuristics For Evaluating Topic Modelsmentioning

confidence: 99%

Identification of Topics and Their Evolution in Management Science: Replicating and Extending an Expert Analysis Using Semi-Automated Methods

Lance¹

View full text Add to dashboard Cite

Latent Dirichlet allocation (LDA) is a popular generative probabilistic model that enables researchers to analyze large semantic datasets; however, few open-source software tools with Graphical User Interfaces (GUIs) are available to researchers. This study identifies an open-source software tool that, in conjunction with a popular electronic spreadsheet software application, can be used to perform topic modeling. A process is developed and evaluated against a pre-existing expert review that examines work published in Management Science on the topics of technological innovation, product development, and entrepreneurship between 1954(Shane and Ulrich, 2004. The process is then replicated using an expanded corpus that includes all articles published in Management Science between 2005 and 2015. The discussion includes an analysis of the process and insights generated by using topic modeling. A replicable process for researchers and suggestions for practitioners are provided.

show abstract

A Computational Approach to Qualitative Analysis in Large Textual Datasets

Cited by 64 publications

References 27 publications

Methods for user‐centered design and evaluation of text analysis tools in a digital history project

Methods for user‐centered design and evaluation of text analysis tools in a digital history project

Who, what and when: how media and politicians shape the Brazilian debate on foreign affairs

Identification of Topics and Their Evolution in Management Science: Replicating and Extending an Expert Analysis Using Semi-Automated Methods

Contact Info

Product

Resources

About