Rapid growth of multi-modal documents on the Internet makes multi-modal summarization research necessary. Most previous research summarizes texts or images separately. Recent neural summarization research shows the strength of the Encoder-Decoder model in text summarization. This paper proposes an abstractive text-image summarization model using the attentional hierarchical Encoder-Decoder model to summarize a text document and its accompanying images simultaneously, and then to align the sentences and images in summaries. A multi-modal attentional mechanism is proposed to attend original sentences, images, and captions when decoding. The DailyMail dataset is extended by collecting images and captions from the Web. Experiments show our model outperforms the neural abstractive and extractive text summarization methods that do not consider images. In addition, our model can generate informative summaries of images.
Summary
Related work is a component of a scientific paper, which introduces other researchers' relevant works and makes comparisons with the current author's work. Automatically generating the related work section of a writing paper provides a tool for researchers to accomplish the related work section efficiently without missing related works. This paper proposes an approach to automatically generating a related work section by comparing the main text of the paper being written with the citations of other papers that cite the same references. Our approach first collects the papers that cite the reference papers of the paper being written and extracts the corresponding citation sentences to form a citation document. It then extracts keywords from the citation document and the paper being written and constructs a graph of the keywords. Once the keywords that discriminate the two documents are determined, the minimum Steiner tree that covers the discriminative keywords and the topic keywords is generated. The summary is generated by extracting the sentences covering the Steiner tree. According to ROUGE evaluations, the experiments show that the citations are suitable for related work generation and our approach outperforms the three baseline methods of MEAD, LexRank, and ReWoS. This work verifies the general summarization method based on connotation and extension through citation.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.