Deep Convolutional Neural Network Textual Features and Multiple Kernel Learning for Utterance-level Multimodal Sentiment Analysis

Abstract

We present a novel way of extracting features from short texts, based on the activation values of an inner layer of a deep convolutional neural network. We use the extracted features in multimodal sentiment analysis of short video clips representing one sentence each. We use the combined feature vectors of textual, visual, and audio modalities to train a classifier based on multiple kernel learning, which is known to be good at heterogeneous data. We obtain 14% performance improvement over the state of the art and present a parallelizable decision-level data fusion method, which is much faster, though slightly less accurate.

Improving deep neural network design with new text data representations

“…CNNs have also been used to aid feature extraction from text. Poria et al [15] built higher level features from textual data, represented by a 306 dimensional vector consisting of a word vector and part of speech values, by training a CNN. They used output of the penultimate fully connected layer to create features to use with other classifiers and found the features produced by the network allowed better classifiers to be trained and using a different learner for classification, such as SVM, performed better than relying on the CNN's final output layer for classification.…”

Sentiment Analysis, Basic Tasks of

“…In the category of supervised methods, [26] employed seed words to guide topic models to learn topics of specific interest to a user, while [63] and [39] employed seeding words to extract related product aspects from product reviews. On the other hand, recent approaches using deep CNNs [17,45] showed significant performance improvement over the state-of-the-art methods on a range of NLP tasks. [17] fed word embeddings to a CNN to solve standard NLP problems such as named entity recognition (NER), part-of-speech (POS) tagging and semantic role labeling.…”

Identifying High Quality Document–Summary Pairs through Text Matching

“…In a previous study [42], deep CNN was used to extract features from short texts for utterance-level multimodal sentiment analysis. Another study [43] also used a seven-layer deep CNN to tag each word in opinionated sentences as either aspect or non-aspect words with the tag results used to extract aspects opinion mining.…”