Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network

Nam, Seonghyeon; Kim, Seon Joo

doi:10.48550/arxiv.1707.08350

Cited by 1 publication

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More comprehensive are works that aim to model the sequence of processing operations that occur within an imaging device [3,13]. Recently, a deep neural network was presented for modeling the scene-dependent color processing of a given camera, where RAW-JPEG image pairs are captured from the camera for training [23]. In our work, we utilize this deep network for modeling color transformations in the imaging pipeline, but infer the model using only a single photograph from an unknown camera.…”

Section: Related Workmentioning

confidence: 99%

“…The structure of our network is illustrated in Figure 3, with the network configuration details given in Table 1. Our networks N 1 and N 2 are adopted from the Multiscale Learnable Histogram network in [23], which achieves state-ofthe-art performance on radiometric calibration. The networks first extract color histogram features from the input image with learnable bin centers and widths.…”

Section: Network Architecturementioning

confidence: 99%

“…Some examples are shown in the Supplementary Material. We additionally utilize the Canon 5D Mark III dataset from [23], which contains 645 RAW-JPEG image pairs of various scenes. Although two additional datasets are provided in [23], we do not use them because we lack access to those camera models for color calibration.…”

Section: Data Collectionmentioning

confidence: 99%

“…For the Sony α-5100 dataset that we collected, we set aside 50 images for testing, use 80% of the remaining images for training, and take the other 20% for validation. For the Canon 5D Mark III dataset, we use the same configuration as in [23], where the ratio between training and validation is 4 : 1, excluding 50 images for testing. Note that none of the images generated using a simulated pipeline were used as test images.…”

Section: Experimental Settingsmentioning

confidence: 99%

“…If the camera that took the photo is available, one could capture a collection of aligned RAW 1 -JPEG image pairs and train a network to map RAW to JPEG [23], as the virtual object is in the RAW domain. However, for broad applicability, it is desirable to composite virtual objects into photos taken by unknown cameras for which we have no training data.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Mimicking the In-Camera Color Pipeline for Camera-Aware Object Compositing

Gao¹,

Li²,

Wang³

et al. 2019

Preprint

View full text Add to dashboard Cite

We present a method for compositing virtual objects into a photograph such that the object colors appear to have been processed by the photo's camera imaging pipeline. Compositing in such a camera-aware manner is essential for high realism, and it requires the color transformation in the photo's pipeline to be inferred, which is challenging due to the inherent one-to-many mapping that exists from a scene to a photo. To address this problem for the case of a single photo taken from an unknown camera, we propose a dual-learning approach in which the reverse color transformation (from the photo to the scene) is jointly estimated. Learning of the reverse transformation is used to facilitate learning of the forward mapping, by enforcing cycle consistency of the two processes. We additionally employ a feature sharing schema to extract evidence from the target photo in the reverse mapping to guide the forward color transformation. Our dual-learning approach achieves object compositing results that surpass those of alternative techniques. * This work was done when Jun Gao was an intern at Microsoft Research.1 RAW: minimally processed image data from the sensor of a camera.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Network Architecturementioning

confidence: 99%