Joint statistical analysis of images and keywords with applications in semantic image enhancement

Abstract

With the advent of social image-sharing communities, millions of images with associated semantic tags are now available online for free and allow us to exploit this abundant data in new ways. We present a fast non-parametric statistical framework designed to analyze a large data corpus of images and semantic tag pairs and find correspondences between image characteristics and semantic concepts. We learn the relevance of different image characteristics for thousands of keywords from one million annotated images. We demonstrate the framework's effectiveness with three different examples of semantic image enhancement: we adapt the gray-level tone-mapping, emphasize semantically relevant colors, and perform a defocus magnification for an image based on its semantic context. The performance of our algorithms is validated with psychophysical experiments.

Keyword-based image color re-rendering with semantic segmentation

“…Our keyword-based image color re-rendering algorithm integrates semantic segmentation with color re-rendering operations. Our method achieves more significant keyword statistics and notably better re-rendering results than the state-of-the-art [1]. indicates the keyword-feature correlation.…”

AutoStyle: Automatic Style Transfer from Image Collections to Users' Images

“…For instance, the sunset filter might produce results tinted orange, blue, or violet, and the user is able to choose among them. In comparison, the method of Lindner et al [LSBS12] only produces one result.…”

“…The work most closely related to ours is by Lindner et al [LSBS12]. They also propose to analyze the style of photos by comparing various features of the photos annotated with a specific keyword.…”

Image aesthetics depends on context

“…Deselaers and Ferrari [12] have shown, by analysing the images in ImageNet [13] that images with semantically similar annotations have more visual attributes in common than images with dissimilar annotations. Lindner et al [14] have also found that images with the same keywords can have features that are statistically significantly different than images that are not annotated with that keyword. We thus investigate if this semantic induced difference in observed features relates also to the aesthetics of an image, or only to its content.…”