A Convolutional Neural Network for Modelling Sentences

Abstract

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline.

Neural Networks for Open Domain Targeted Sentiment

“…They further extend the method to a syntactic treebank annotated with sentiment labels (Socher et al, 2013). More recently, Kalchbrenner et al (2014) use a dynamic pooling network to include the structure…”

SwissCheese at SemEval-2016 Task 4: Sentiment Classification Using an Ensemble of Convolutional Neural Networks with Distant Supervision

“…Deep neural networks have shown great promises at capturing salient features for these complex tasks (Mikolov et al, 2013b;Severyn and Moschitti, 2015a). Particularly successful for sentiment classification were Convolutional Neural Networks (CNN) (Kim, 2014;Kalchbrenner et al, 2014;Severyn and Moschitti, 2015a;Severyn and Moschitti, 2015b;Johnson and Zhang, 2015), on which our work builds upon. * These authors contributed equally to this work These networks typically have a large number of parameters and are especially effective when trained on large amounts of data.…”

The DCU Discourse Parser: A Sense Classification Task

“…Although a paragraph vector did not work efficiently, our model has a tentative model which does not have interaction between relational, paragraph, and word embeddings such as in (Denil et al, 2015), which is one immediate challenge. Then, other challenge includes replacement of a paragraph vector model with a convolutional sentence vector model (Kalchbrenner et al, 2014) and RNN-LSTM model (Le and Zuidema, 2015). The former approach is related to the supervised learning instead of unsupervised learning.…”