Transient attributes for high-level understanding and editing of outdoor scenes

Abstract

more "winter" more "night" more "warm" more "moist" more "rain" more "autumn" Figure 1: Our method enables high-level editing of outdoor photographs. In this example, the user provides an input image (left) and six attribute queries corresponding to the desired changes, such as more "autumn". Our method hallucinates six plausible versions of the scene with the desired attributes (right), by learning local color transforms from a large dataset of annotated outdoor webcams. AbstractWe live in a dynamic visual world where the appearance of scenes changes dramatically from hour to hour or season to season. In this work we study "transient scene attributes" -high level properties which affect scene appearance, such as "snow", "autumn", "dusk", "fog". We define 40 transient attributes and use crowdsourcing to annotate thousands of images from 101 webcams. We use this "transient attribute database" to train regressors that can predict the presence of attributes in novel images. We demonstrate a photo organization method based on predicted attributes. Finally we propose a high-level image editing method which allows a user to adjust the attributes of a scene, e.g. change a scene to be "snowy" or "sunset". To support attribute manipulation we introduce a novel appearance transfer technique which is simple and fast yet competitive with the state-of-the-art. We show that we can convincingly modify many transient attributes in outdoor scenes.

Discovering states and transformations in image collections

“…Each noun can be in a variety of adjective states, e.g., rope can be short, long, coiled, etc. Surprisingly little previous work in computer vision has focused on adjectives [11,17].…”

Lighting transfer across multiple views through local color transforms

“…Shih et al [16] successfully synthesize different-time-ofday images by learning color transformations from time-lapse videos. A similar approach by Laffont et al [17] enables appearance transfer of time-of-day, weather, or season by observing color changes in a webcam database. However, both methods rely on the availability of images of different appearance from the same webcam.…”

“…We express the transform A k for k as a linear matrix that maps the color of a pixel in the source image S to another pixel in the target T . We learn the local transforms as linear models [17] in RGB color space. The local color transforms are modeled as the solutions to an optimization problem: arg min…”

Time-lapse mining from internet photos

“…Scene geometry can also be inferred from the shadows cast by clouds [Jacobs et al 2010] or by finding correspondences along the shadow edges [Abrams et al 2013]. By using a database of time-lapse videos, [Shih et al 2013;Laffont et al 2014] learn appearance transfer models that can change the time of day or time of year of a photograph.…”