FantasyCoref: Coreference Resolution on Fantasy Literature Through Omniscient Writer’s Point of View

Abstract

This paper presents a new corpus and annotation guideline for a novel coreference resolution task on fictional texts, and analyzes its unique characteristics. FantasyCoref contains 211 stories of Grimms' Fairy Tales and 3 other fantasy literature annotated in the omniscient writer's point of view (OWV) to handle distinctive aspects in this genre. This task is more challenging than general coreference resolution in two ways. First, documents in our corpus are 2.5 times longer than the ones in OntoNotes, raising a new layer of difficulty in resolving long-distant referents. Second, annotation of literary styles and concepts raise several issues which are not sufficiently addressed in the existing annotation guidelines. Hence, considerations on such issues and the concept of OWV are necessary to achieve high interannotator agreement (IAA) in coreference resolution of fictional texts. We carefully conduct annotation tasks in four stages to ensure the quality of our annotation. As a result, a high IAA score of 87% is achieved using the standard coreference evaluation metric. Finally, state-of-the-art coreference resolution approaches are evaluated on our corpus. After training with our annotated dataset, there was a 2.59% and 3.06% improvement over the model trained on the OntoNotes dataset. Also, we observe that the portion of errors specific to fictional texts declines after the training.

Big data meets storytelling: using machine learning to predict popular fanfiction

“…A co-reference resolution system identifies characters across multiple words, thus enabling greater textual and character analysis. Several tools have been proposed for co-reference solution, such as Yang's use of neural networks (LSTM) trained on Jane Austen's Sense and Sensibility (Yang 2022), or FantasyCoref, trained on Grimm's Fairy Tales, Alice's Adventures in Wonderland, and two stories from the Arabian Nights (Han et al 2021). In the case of fanfiction, the specialized tool is FanfictionNLP (Yoder et al 2021).…”

Portrayal: Leveraging NLP and Visualization for Analyzing Fictional Characters

“…Here "he" refers to "Gary." Identifying such coreferences and clustering them correctly for a story is challenging as book length coreference resolution is an open problem in NLP [27]. However, this is critical for our case as extraction of traits such as actions and emotions will depend on correct identification of where the characters appear in the text.…”

“…Human-in-the-loop Co-reference Resolution. Book-length co-reference resolution was a major challenge for our work as it is still an unsolved problem in NLP [27]. Popular libraries such as bookNLP [3] use the imperfect co-reference models and provide disclaimers that the outcome will be error-prone.…”