Images2Poem in different contexts with Dual‐CharRNN

Abstract

Image to caption has attracted extensive research attention recently. However, image to poetry, especially Chinese classical poetry, is much more challenging. Previous works mainly focus on generating coherent poetry without taking the contexts of poetry into account. In this paper, we propose an Images2Poem with the Dual‐CharRNN model which exploits images to generate Chinese classical poems in different contexts. Specifically, we first extract a few keywords representing elements from the given image based on multi‐label image classification. Then, these keywords are expanded to related ones with the planning‐based model. Finally, we employ Dual‐CharRNN to generate Chinese classical poetry in different contexts. A comprehensive evaluation of human judgements demonstrates that our model achieves promising performance and is effective in enhancing poetry's semantic consistency, readability, and aesthetics. We present an Images2Poem with the Dual‐CharRNN model exploiting images to generate Chinese classical poems in different contexts, which effectively improves the semantic consistency, readability and aesthetics of the generated poetry.