Estimating the Reflectance Property from Refocused Images and its Application to Auto Material Appearance Balancing

Abstract

In this paper, we propose a method to estimate the reflectance property from refocused images from a light source reflected on an object. The blurred information of the light source on the surface of the object is expected to be a practical method to estimate the reflectance property, although various methods have already been proposed. Because the degree of blurred information is changed with the position of focus in the camera, we introduce a light field camera that can change the position of focus after the image is captured. In this research, we chose the refocused image where the light source is focused through the reflection on the object surface. Based on the blurred information of the focused light source, we estimate the reflectance property of the object. The estimated reflectance property is applied to inverse rendering for auto material appearance balancing.

Visual Cortex based Material Appearance Transfer Model

“…As a framework for material perception, N. Tsumura initiated the skin color appearance and proposed the concept of appearance delivering system [2]. IS&T In Brain Information Science research on SHITSUKAN (material perception) [3] by MEXT (Ministry of Education, Culture, Sports, Science and Technology) in Japan, the first stage (2010-2014 led by Dr. H. Komatsu) has concluded and the second stage (2015-2019 led by Dr. S. Nishida) stepped forward into ''multi-dimensional'' material perception and currently, further research on ''deep texture'' is progressing.…”