Chapter Captor: Text Segmentation in Novels

Abstract

Books are typically segmented into chapters and sections, representing coherent subnarratives and topics. We investigate the task of predicting chapter boundaries, as a proxy for the general task of segmenting long texts. We build a Project Gutenberg chapter segmentation data set of 9,126 English novels, using a hybrid approach combining neural inference and rule matching to recognize chapter title headers in books, achieving an F1-score of 0.77 on this task. Using this annotated data as ground truth after removing structural cues, we present cut-based and neural methods for chapter segmentation, achieving an F1-score of 0.453 on the challenging task of exact break prediction over book-length documents. Finally, we reveal interesting historical trends in the chapter structure of novels.

Paragraph-level attention based deep model for chapter segmentation

“…Text segmentation ( Pethe, Kim & Skiena, 2020 ; Haruechaiyasak, Kongyoung & Dailey, 2008 ; Koshorek et al, 2018 ; Li et al, 2020 ; Lukasik et al, 2020 ; Nguyen et al, 2021 ) is a method that is typically used to segment chapters by separating text into multiple segments or boundaries. It has also been used in many natural language processing tasks, such as word tokenization, text summarization ( Hulliyah & Kusuma, 2010 ; Awasthi et al, 2021 ), question answering prediction ( Wang, Ling & Hu, 2019 ), and machine translation ( Kong, Zhang & Hovy, 2020 ; Gupta et al, 2021 ; Budiwati & Aritsugi, 2022 ).…”

“…Recent research has proposed the building of a deep-learning system to automatically identify chapter boundaries. For example, Pethe, Kim & Skiena (2020) proposed a Chapter Captor used to correctly recognize chapter breakpoints in novels. They proposed using Bidirectional Encoder Representations from Transformers ( Devlin et al, 2018 ) to learn the semantic features and generate token-wise softmax probabilities.…”

“…Considering these limitations of BERT ( Pethe, Kim & Skiena, 2020 ), we instead propose the use of the XLNet model ( Yang et al, 2019 ). The XLNet model is a generalized auto-regressive model that uses a permutation language model that helps the model learn a bidirectional context.…”

“…The paragraph-level attention model had an F1 score of 0.8084, while the ensemble method improved its F1 score to 0.8856. Our results were then compared with the best methods found in Pethe, Kim & Skiena (2020) (BERT Break Point Prediction Model) and the F1 confusion matrix, as Their’s methods were considered to be the best practice. Pethe, Kim & Skiena (2020) used a pre-trained BERT model for the Next Sentence Prediction task combining with the dynamic programming algorithm.…”

“…Our results were then compared with the best methods found in Pethe, Kim & Skiena (2020) (BERT Break Point Prediction Model) and the F1 confusion matrix, as Their’s methods were considered to be the best practice. Pethe, Kim & Skiena (2020) used a pre-trained BERT model for the Next Sentence Prediction task combining with the dynamic programming algorithm. The BERT Break Point Prediction model ( Pethe, Kim & Skiena, 2020 ) successfully competed with all baseline models, including the C99 algorithm from Choi (2000) , the three-layer baseline perceptron model with 300 neurons in each layer ( Badjatiya et al, 2018 ) and trained word2vec embeddings from Mikolov et al (2013) , and the neural model described by Badjatiya et al (2018) that used long short-term memory (LSTM).…”