How quantifying the shape of stories predicts their success

Abstract

Narratives, and other forms of discourse, are powerful vehicles for informing, entertaining, and making sense of the world. But while everyday language often describes discourse as moving quickly or slowly, covering a lot of ground, or going in circles, little work has actually quantified such movements or examined whether they are beneficial. To fill this gap, we use several state-of-the-art natural language-processing and machine-learning techniques to represent texts as sequences of points in a latent, high-dimensional semantic space. We construct a simple set of measures to quantify features of this semantic path, apply them to thousands of texts from a variety of domains (i.e., movies, TV shows, and academic papers), and examine whether and how they are linked to success (e.g., the number of citations a paper receives). Our results highlight some important cross-domain differences and provide a general framework that can be applied to study many types of discourse. The findings shed light on why things become popular and how natural language processing can provide insight into cultural success.

Divergent Semantic Integration (DSI): Extracting Creativity from Narratives with Distributional Semantic Modeling

“…These results demonstrate impressive predictive power of w2w SemDiv in capturing creativity in narrative text. As previous work shows minimal predictive power for automated assessments of creativity in narrative text, these results represent a substantial step forward for researchers from diverse disciplines including psychology (D 'Souza, 2021;Toubia et al, 2021;Zedelius et al, 2019), linguistics (Mozaffari, 2013), education (Graham et al, 2002;Vaezi et al, 2019), and creative writing (Bland, 2011).…”

“…Most prior work in validating the use of automating language analysis for creativity assessment has focused on word association tasks and the alternate uses task of divergent thinking (Beaty & Johnson, 2021;Dumas et al, 2020;Gray et al, 2019;Prabhakaran et al, 2014). However, the automated creativity assessments in prior work that focused on narrative text did not demonstrate strong capacity to predict humanrated creativity (Toubia et al, 2021;Zedelius et al, 2019). In addition, our w2w SemDiv metric is among the first automated creativity metrics to focus on idea diversity (see Consideration of the socio-cultural context in the automated assessment of creativity is critical to establishing "algorithmic-fairness".…”

“…Although some indices explained variance in imagery and voice, neither Coh-Metrix or LIWC metrics explained meaningful variance in originality. Toubia et al (2021) developed a number of novel computational metrics to predict how highly humans rated movies, T.V. shows, and books.…”

“…Semantic diversity offers a way to quantify the semantic distance between ideas in a story. Semantic diversity refers to the average distance between the contexts in a text and there are numerous ways to quantify it (Cevoli et al, 2021;Johns, 2021;Hoffman et al, 2013; see volume from Toubia et al, 2021). We propose that averaging the semantic distances between all words in a story parsimoniously captures the distance between ideas.…”

Disclosing the relationship between citation structure and future impact of a publication

“…Their experimental results disclosed that a citation time window of 3 years can achieve acceptable accuracy in predicting the long‐term impact of publications, and the impact factor (i.e., the impact factor becomes negligible only 2 years after publication). Toubia et al (2021) proposed three metrics (speed, volume, and circuitousness) to quantify the semantic progression of texts, and confirmed that these metrics have a significant impact on the long‐term impact of a paper by a least absolute shrinkage and selection operator (Lasso) regression. In addition, deep learning algorithms have also achieved fruitful results in citation count prediction.…”