Proceedings of the Workshop on Intelligent Interactive Systems and Language Generation (2IS&NLG)

Abstract

In this paper, we propose an end-toend CNN-LSTM model for generating descriptions for sequential images with a local-object attention mechanism. To generate coherent descriptions, we capture global semantic context using a multilayer perceptron, which learns the dependencies between sequential images. A paralleled LSTM network is exploited for decoding the sequence descriptions. Experimental results show that our model outperforms the baseline across three different evaluation metrics on the datasets published by Microsoft.