Deep Highlight Removal Using Temporal Dark Prior in High-Speed Domain

Abstract

This paper proposes a deep highlight removal method based on the dichromatic model under alternating current (AC) light sources with a new diffuse prior on temporal domain, temporal dark prior. An input image is decomposed into specular and diffuse components using a deep network. Due to AC powered lights, both incident and reflected lights are time-varying. We exploit the periodic variation property of the specular and diffuse reflections as a prior for dichromatic model based image decomposition. In addition, we propose a new temporal dark prior as a pseudo-diffuse reflection. Unlike the conventional prior in the spatial domain, to the best of our knowledge, this is the first study to utilize a diffuse prior on the temporal domain for highlight removal. The blurred version of the temporal dark prior is additionally fed to the network to alleviate hole artifacts. It is demonstrated through diverse experiments that these temporal priors can strongly contribute to accurate image decomposition, leading to better highlight removal. INDEX TERMS Dichromatic model, highlight removal, high-speed camera.The associate editor coordinating the review of this manuscript and approving it for publication was Yongjie Li.

Joint Deep Estimation of Intrinsic and Dichromatic Image Decomposition

“…There are few studies that have exploited temporal features for dichromatic decomposition. The works in [6], [20], and [52] use the intensity fluctuation of AC lights captured in high speed video. Tsuji [20] assumes the linear relation between minimum and maximum luminance of a high-speed video.…”

“…Yoo et al [6] proposed a deep network that estimates all parameters of the dichromatic model including illuminant color. Ha et al [52] introduced a new temporal dark prior for dichromatic model based decomposition. However, Tsuji [20] still showed color distortion on strong highlight regions which is a common problem of highlight removal.…”