What Happens Next? Event Prediction Using a Compositional Neural Network Model

Abstract

We address the problem of automatically acquiring knowledge of event sequences from text, with the aim of providing a predictive model for use in narrative generation systems. We present a neural network model that simultaneously learns embeddings for words describing events, a function to compose the embeddings into a representation of the event, and a coherence function to predict the strength of association between two events. We introduce a new development of the narrative cloze evaluation task, better suited to a setting where rich information about events is available. We compare models that learn vector-space representations of the events denoted by verbs in chains centering on a single protagonist. We find that recent work on learning vector-space embeddings to capture word meaning can be effectively applied to this task, including simple incorporation of a verb's arguments in the representation by vector addition. These representations provide a good initialization for learning the richer, compositional model of events with a neural network, vastly outperforming a number of baselines and competitive alternatives.

Event Representations With Tensor-Based Compositions

“…For comparison, we also report on an automatically generated MCNC task. Since we do not restrict the evaluation to narrative chains with a single protagonist, the numbers for the automatic MCNC task are lower than those reported in Granroth-Wilding and Clark (2016).…”

“…This basic architecture was used to generate event representations for narrative cloze tasks (Modi and Titov 2013;Modi 2016;Granroth-Wilding and Clark 2016). We adapt this architecture for our task -given the input event, whose representation is to be composed, predict the neighboring context (other events or words in the sentence).…”

What Happens Next? Future Subevent Prediction Using Contextual Hierarchical LSTM

“…For example, Radinsky et al (2012) extracted generalized causality relations of two events in the form of "x causes y" from past news and applied the templates on a present news event to predict the next possible event. Granroth-Wilding et al (2016) extract event knowledge about typical sequences of events from text (2008) and learned the coherence score of two events using a compositional neural network. They aim to predict the strength of association between two events, which could be used for predicting whether an event is likely to be the next event.…”

“…For example, Radinsky et al (2012) extracted causality relations between two events and generalized them using ontology for prediction. Granroth-Wilding et al (2016) extracted event chains (Chambers and Jurafsky 2008) from texts and learned the coherence score of two events using a compositional neural network. Manshadi et al (2008) learned a probabilistic language model of the event sequences.…”

FEEL: Featured Event Embedding Learning

“…As the example above suggests, predicting "what happens next? ", also known as the Narrative Cloze (NC) task (Chambers and Jurafsky 2008;Granroth-Wilding and Clark 2016), is the preferred way of evaluating such models. In this paper, we propose a generalization of this task highlighting the importance of evaluating inferences over chains of future events.…”