Automated Storytelling via Causal, Commonsense Plot Ordering

Ammanabrolu, Prithviraj; Cheung, Wesley; Broniec, William; Riedl, Mark O.

doi:10.1609/aaai.v35i7.16733

Cited by 33 publications

(17 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…; Peng et al (2021) explore using external knowledge such as commonsense for story generation. Xu et al (2020); Ammanabrolu et al (2021) combine storyline planning and commonsense reasoning. We find that although there are studies which explore the use of GPT3 for story generation.…”

Section: Related Workmentioning

confidence: 99%

Can Very Large Pretrained Language Models Learn Storytelling With A Few Examples?

Xie¹,

Cohn²,

Lau³

2023

Preprint

View full text Add to dashboard Cite

While pre-trained language models can generate individually fluent sentences for automatic story generation, they struggle to generate stories that are coherent, sensible and interesting. Current state-of-the-art (SOTA) story generation models explore using higher-level features such as plots or commonsense knowledge to improve the quality of generated stories. Prompt-based learning using very large pre-trained language models (VLPLMs) such as GPT3 has demonstrated impressive performance even across various NLP tasks. In this paper, we present an extensive study using automatic and human evaluation to compare the story generation capability of VLPLMs to those SOTA models in three different datasets where stories differ in style, register and length. Our results show that VLPLMs generate much higher quality stories than other story generation models, and to a certain extent rival human authors, although preliminary investigation also reveals that they tend to "plagiarise" real stories in scenarios that involve world knowledge.

show abstract

Section: Related Workmentioning

confidence: 99%

Can Very Large Pretrained Language Models Learn Storytelling With A Few Examples?

Xie¹,

Cohn²,

Lau³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…[25] extracted the representation from COMET to improve the commonsense reasoning of the model. [26] used a generation model fine-tuned on the commonsense knowledge base to complete the tasks.…”

Section: Commonsense Knowledge Integrated Generationmentioning

confidence: 99%

CEG: A joint model for causal commonsense events enhanced story ending generation

Zhang

Yang

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

With the success of pre-trained language models, the performance of story ending generation has been dramatically improved while remaining challenging due to the lack of commonsense reasoning ability. Most previous works mainly focus on using commonsense knowledge to enhance the implicit correlations between words but ignore the hidden causality of sentences or events. In this paper, we propose Causal commonsense Enhanced joint model for story ending Generation (CEG), which incorporates causal commonsense events knowledge to generate a reasonable story ending. Specifically, we first develop a commonsense events inference model trained on GLUCOSE, which converts static knowledge into a dynamic generation model to discover unseen knowledge. It uses prompts to produce various commonsense events behind the stories as pseudo-labels of the dataset. Then, we propose a joint model for the causal events inference task and the story ending generation task to inject inference knowledge into the generation, which consists of a shared encoder, an inference decoder, and a generation decoder. In the causal events inference task, we use the shared encoder and the inference decoder to reason the causal events behind each sentence of the story context to help the model better understand the story and provide long-distance dependencies for the story ending generation. In story ending generation, we combine the hidden states of the causal events with the story context to generate the story ending by the shared encoder and the generation decoder. We jointly train the model on two tasks so that the generation decoder produces the story endings that better match the clues. Experimental results on the ROCStories dataset show that our model outperforms the previous works, demonstrating the effectiveness of the joint model and the generated causal events.

show abstract

“…Each code corresponds to a fixed-length span, which does not always agree with real text structures and makes it hard to model specific semantic dependencies. Some studies tried to incorporate external knowledge or reasoning models to guide commonsense story generation (Guan et al 2020;Xu et al 2020;Ammanabrolu et al 2021), which may lack generalization to other domains such as news. Another line improved coherence by learning high-level representations of prefix sentences (Li, Luong, and Jurafsky 2015;Guan et al 2021), which does not emphasize the central role of entities.…”

Section: Related Workmentioning

confidence: 99%

Generating Coherent Narratives by Learning Dynamic and Discrete Entity States with a Contrastive Framework

Guan

Yang

Zhang

et al. 2023

AAAI

View full text Add to dashboard Cite

Despite advances in generating fluent texts, existing pretraining models tend to attach incoherent event sequences to involved entities when generating narratives such as stories and news. We conjecture that such issues result from representing entities as static embeddings of superficial words, while neglecting to model their ever-changing states, i.e., the information they carry, as the text unfolds. Therefore, we extend the Transformer model to dynamically conduct entity state updates and sentence realization for narrative generation. We propose a contrastive framework to learn the state representations in a discrete space, and insert additional attention layers into the decoder to better exploit these states. Experiments on two narrative datasets show that our model can generate more coherent and diverse narratives than strong baselines with the guidance of meaningful entity states.

show abstract

Automated Storytelling via Causal, Commonsense Plot Ordering

Cited by 33 publications

References 25 publications

Can Very Large Pretrained Language Models Learn Storytelling With A Few Examples?

Can Very Large Pretrained Language Models Learn Storytelling With A Few Examples?

CEG: A joint model for causal commonsense events enhanced story ending generation

Generating Coherent Narratives by Learning Dynamic and Discrete Entity States with a Contrastive Framework

Contact Info

Product

Resources

About