Natural-Language-Based Conversion of Images to Mobile Multimedia Experiences

Abstract

Abstract. We describe an approach for viewing any large, detail-rich picture on a small display by generating a video from an the image, as taken by a virtual camera moving across it at varying distance. Our main innovation is the ability to build the virtual camera's motion from a textual description of a picture, e.g., a museum caption, so that relevance and ordering of image regions are determined by co-analyzing image annotations and natural language text. Furthermore, our system arranges the resulting presentation such that it is synchronized with an audio track generated from the text by use of a text-to-speech system.