more "winter" more "night" more "warm" more "moist" more "rain" more "autumn" Figure 1: Our method enables high-level editing of outdoor photographs. In this example, the user provides an input image (left) and six attribute queries corresponding to the desired changes, such as more "autumn". Our method hallucinates six plausible versions of the scene with the desired attributes (right), by learning local color transforms from a large dataset of annotated outdoor webcams. AbstractWe live in a dynamic visual world where the appearance of scenes changes dramatically from hour to hour or season to season. In this work we study "transient scene attributes" -high level properties which affect scene appearance, such as "snow", "autumn", "dusk", "fog". We define 40 transient attributes and use crowdsourcing to annotate thousands of images from 101 webcams. We use this "transient attribute database" to train regressors that can predict the presence of attributes in novel images. We demonstrate a photo organization method based on predicted attributes. Finally we propose a high-level image editing method which allows a user to adjust the attributes of a scene, e.g. change a scene to be "snowy" or "sunset". To support attribute manipulation we introduce a novel appearance transfer technique which is simple and fast yet competitive with the state-of-the-art. We show that we can convincingly modify many transient attributes in outdoor scenes.
Figure 1: Our method leverages the heterogeneity of photo collections to automatically decompose photographs of a scene into reflectance and illumination layers. The extracted reflectance layers are coherent across all views, while the illumination captures the shading and shadow variations proper to each picture. Here we show the decomposition of three photos in the collection. AbstractAn intrinsic image is a decomposition of a photo into an illumination layer and a reflectance layer, which enables powerful editing such as the alteration of an object's material independently of its illumination. However, decomposing a single photo is highly under-constrained and existing methods require user assistance or handle only simple scenes. In this paper, we compute intrinsic decompositions using several images of the same scene under different viewpoints and lighting conditions. We use multi-view stereo to automatically reconstruct 3D points and normals from which we derive relationships between reflectance values at different locations, across multiple views and consequently different lighting conditions. We use robust estimation to reliably identify reflectance ratios between pairs of points. From these, we infer constraints for our optimization and enforce a coherent solution across multiple views and illuminations. Our results demonstrate that this constrained optimization yields high-quality and coherent intrinsic decompositions of complex scenes. We illustrate how these decompositions can be used for image-based illumination transfer and transitions between views with consistent lighting.
In this appendix, we describe the details of the illuminant calibration step for the sky and the sun.First, because our model separates sun light from sky light, we need to remove sun pixels from the environment map. We define the sun position as the barycenter of the saturated sun pixels, and use inpainting to fill-in these saturated pixels from their neighbors. Since our model also separates sky light from indirect light, we use a standard color selection tool to label sky pixels that will contribute to the sky illumination, while other pixels (building, trees) will contribute to indirect lighting. This is illustrated in Fig. 5c. Second, we align the environment map and sun with the reconstructed scene. To do so we manually mark a vertical edge of the reconstructed geometry and rotate the environment map and sun until the cast shadow of the virtual edge is aligned with that in the photograph.Finally, the environment map only captures a scaled version of the scene lighting since the sphere is not perfectly specular. We need to compensate for this scaling factor in each color channel.In our system the environment map is used to compute both the sky illumination S sky and part of the indirect illumination S ind , the other part being computed from the geometric proxy (see Section 5 for more details). We estimate the color transfer function of the reflective sphere K (represented as a RGB vector) by taking a photograph of a neutral gray card with known reflectance R placed in sun shadow. We intentionally place the card at a position where we expect its geometry to be well reconstructed. From the image formation model we havewhere S env .denotes the illumination terms computed from the environment map and S proxy .the ones computed from the geometric proxy. We use this equation to solve for the unknown K.• REVES / INRIA Sophia AntipolisWe similarly recover the sun radiance L sun by taking a second picture of the gray card placed in sunlight (v sun = 1). From this picture we havewhere L sun is the only unknown.We could envisage other approaches for this calibration. Cabral et al.[1] use a compass to compute the environment map orientation with respect to the view direction in each photograph. In the work of Yu and Malik [2], a set of photographs of the horizon are taken, and a sky model is fitted. This involves taking several photographs, as opposed to the only two we take of the sphere, and also requires the use of two neutral lens filters to estimate sun radiance. In addition, fitting parameters of sky models can require specific tuning for relighting applications, as was the case for morning and evening sky in the work of Cabral et al. [1]. SUPPLEMENTARY VIDEOA supplementary video is attached. It illustrates some applications of our decomposition in image editing software. SUPPLEMENTARY FIGURESIn this document we show results on four scenes, for a total of 14 views. For each viewpoint, we provide the input (rectified) image, estimated reflectance, as well as the constraints and estimated total illumination, sun i...
We introduce a method to compute intrinsic images for a multi-view set of outdoor photos with cast shadows, taken under the same lighting. We use an automatic 3D reconstruction from these photos and the sun direction as input and decompose each image into reflectance and shading layers, despite the inaccuracies and missing data of the 3D model. Our approach is based on two key ideas. First, we progressively improve the accuracy of the parameters of our image formation model by performing iterative estimation and combining 3D lighting simulation with 2D image optimization methods. Second we use the image formation model to express reflectance as a function of discrete visibility values for shadow and light, which allows us to introduce a robust visibility classifier for pairs of points in a scene. This classifier is used for shadow labelling, allowing us to compute high quality reflectance and shading layers. Our multi-view intrinsic decomposition is of sufficient quality to allow relighting of the input images. We create shadow-caster geometry which preserves shadow silhouettes and using the intrinsic layers, we can perform multi-view relighting with moving cast shadows. We present results on several multi-view datasets, and show how it is now possible to perform image-based rendering with changing illumination conditions.
We present a method for intrinsic image decomposition, which aims to decompose images into reflectance and shading layers. Our input is a sequence of images with varying illumination acquired by a static camera, e.g. an indoor scene with a moving light source or an outdoor timelapse. We leverage the local color variations observed over time to infer constraints on the reflectance and solve the ill-posed image decomposition problem. In particular, we derive an adaptive local energy from the observations of each local neighborhood over time, and integrate distant pairwise constraints to enforce coherent decomposition across all surfaces with consistent shading changes. Our method is solely based on multiple observations of a Lambertian scene under varying illumination and does not require user interaction, scene geometry, or an explicit lighting model. We compare our results with several intrinsic decomposition methods on a number of synthetic and captured datasets.
Image-based rendering (IBR)
Der Burgwall von Décines (Rhône) ist einer der zahlreichen, von einem Graben umgebenen Anlagen, die das Velin (ein kleines Gebiet im ôstlichen Lyonnaiser Raum) im Mittelalter aufweist. Durch eine relativ ausgedehnte Grabung konnten mehrere Abschnitte beim Bau der Befestigungen entdeckt werden, ausgehend von einer mit einem Graben umgebenen Ringmauer bis zum klassischen Burgwall, der ein natùrliches Relief mit einbe-zieht : Anhôhe, Wirtschaftshof sowie eine zweifache Grabenreihe mit einem Wall in der Mitte. Zumindest drei aufeinander-folgende Benutzungsphasen sind auf dem Wirtschaftshof zu erkennen. Die âlteste stellt eine kleine metallverarbeitende Werkstatt dar, die anderen sind Uberreste von Holz-und Strohlehmbehausungen. LâBt man eine Weidenutzung der Anlage am Ende des Mittelalters auBer acht, so erfuhr der Verteidigungsbau nur eine verhàltnismàBig kurze Nutzung (Ende X. Jh.-Anfang XI. Jh.). Eine recht reichliche Ausstattung (Keramik, Metall, Fauna) erschlieBt uns die Tâtigkei-ten der Bewohner des Burgwalles, der zweifellos zur Verteidigung diente. Die Zeugnisse der Metallverarbeitung, indes auch Textilien und landwirtschaftliche Fundstiicke lassen vermuten, daB der an einem strategischen Punkt errichtete Burgwall von Décines ebenfalls fur die Inwertsetzung des Velin eine groBe Rolle spielt.
Intrinsic images aim at separating an image into its reflectance and illumination components to facilitate further analysis or manipulation. This separation is severely ill-posed and the most successful methods rely on user indications or precise geometry to resolve the ambiguities inherent to this problem. In this paper we propose a method to estimate intrinsic images from multiple views of an outdoor scene without the need for precise geometry and with a few manual steps to calibrate the input. We use multiview stereo to automatically reconstruct a 3D point cloud of the scene. Although this point cloud is sparse and incomplete, we show that it provides the necessary information to compute plausible sky and indirect illumination at each 3D point. We then introduce an optimization method to estimate sun visibility over the point cloud. This algorithm compensates for the lack of accurate geometry and allows the extraction of precise shadows in the final image. We finally propagate the information computed over the sparse point cloud to every pixel in the photograph using image-guided propagation. Our propagation not only separates reflectance from illumination, but also decomposes the illumination into a sun, sky and indirect layer. This rich decomposition allows novel image manipulations as demonstrated by our results.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.