Text Semantics Capture Political and Economic Narratives

Heroes, Villains, and Victims, and GPT-3: Automated Extraction of Character Roles Without Training Data

“…On the broader problem of extracting stereotypical character roles, prior work has explored a variety of methods, including the detection of personas using annotated data combined with feature engineering and regression Bamman et al [2013b]; parsing and pattern matching tools to identify a consistent set of personas (e.g. doctor, nurse, doula) across testimonials about childbirth and then assess the relative power dynamics Antoniak et al [2019]; annotations of German news and social media sentences for villains and rogues and transformer models to machine-tag these roles Klenner et al [2021]; clustering of structural plot information from folktales Jahan et al [2021]; and a combination of NER and clustered phrase embeddings to identify repeatedly occurring entities, along with semantic role labeling to identify how entities are connected by actions Ash et al [2021]. Our method does not rely on labeled data, but we employ some of these techniques (e.g., clustering) to support the legibility of our results.…”

“…We have hundreds of unique answers for each role, with many singletons. To reduce the dimensionality of these outputs and make them more interpretable, we encode the phrases using S- BERT Reimers and Gurevych [2019] and apply k-means clustering to the resulting vectors Jahan et al [2021], Ash et al [2021]. After manual inspection for different k, we select k=20.…”

“…The queries for our relatively small-scale analysis of state-of-the-union address speeches cost nine dollars using the 13B-parameter GPT-3 model. Scaling this up to larger corpora of thousands or millions of documents, such as the U.S. Congressional Record Ash et al [2021], would be prohibitively expensive. Hence, using even larger pre-trained models, such as PaLM Chowdhery et al [2022], is likely not cost-effective for most academic research.…”

Proceedings of the 4th Workshop of Narrative Understanding (WNU2022)

“…On the broader problem of extracting stereotypical character roles, prior work has explored a variety of methods, including the detection of personas using annotated data combined with feature engineering and regression (Bamman et al, 2013b); parsing and lexical matching tools to identify a consistent set of personas (e.g. doctor, nurse, doula) across testimonials about childbirth and then assess the relative power dynamics ; annotations of German news and social media sentences for villains and rogues and transformer models to machine-tag these roles (Klenner et al, 2021); clustering of structural plot information from folktales (Jahan et al, 2021); and a combination of NER and clustered phrase embeddings to identify repeatedly occurring entities, along with semantic role labeling to identify how entities are connected by actions (Ash et al, 2021). Our method does not rely on labeled data, but we employ some of these techniques (e.g., clustering) to support the legibility of our results.…”

“…We have hundreds of unique answers for each role, with many singletons. To reduce the dimensionality of these outputs and make them more interpretable, we encode the phrases using S-BERT and apply k-means clustering to the resulting vectors (Jahan et al, 2021;Ash et al, 2021). After manual inspection for different k, we select k=20.…”