Big open-source social science: capabilities and methodology for automating social science analytics

Palladino, Anthony; Bienenstock, Elisa Jayne; George, C.; Moore, Kendra E.

doi:10.1117/12.2306500

Cited by 2 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We then binned documents by month of publication; in practice, time bin size could be chosen based on the frequency of documents under investigation. For each n-gram, g, over m time units, we generated a relative usage trend of m values by calculating the proportion of observed n-grams in each time bin that are instances of g. We used the Big Open-Source Social Science (BOSSS) tool [9,10] to collect a corpus of 2,760 news articles from Google News discussing Malindi, a city on the Kenyan coast, published between January 2016 and September 2018. As a validation technique for our usage quantification, we identified particular events the corpus discusses that we expected to associate with use of certain n-grams and ultimately with PMESII-ASCOPE topics.…”

Section: Introductionmentioning

confidence: 99%

An NLP approach to quantify dynamic salience of predefined topics in a text corpus

Bock¹,

Palladino²,

Smith-Heisters³

et al. 2021

Preprint

View full text Add to dashboard Cite

The proliferation of news media available online simultaneously presents a valuable resource and significant challenge to analysts aiming to profile and understand social and cultural trends in a geographic location of interest. While an abundance of news reports documenting significant events, trends, and responses provides a more democratized picture of the social characteristics of a location, making sense of an entire corpus to extract significant trends is a steep challenge for any one analyst or team. Here, we present an approach using natural language processing techniques that seeks to quantify how a set of predefined topics of interest change over time across a large corpus of text. We found that, given a predefined topic, we can identify and rank sets of terms, or n-grams, that map to those topics and have usage patterns that deviate from a normal baseline. Emergence, disappearance, or significant variations in n-gram usage present a ground-up picture of a topic's dynamic salience within a corpus of interest.

show abstract

Section: Introductionmentioning

confidence: 99%