Disentangled Image Matting

Abstract

Most previous image matting methods require a roughlyspecificed trimap as input, and estimate fractional alpha values for all pixels that are in the unknown region of the trimap. In this paper, we argue that directly estimating the alpha matte from a coarse trimap is a major limitation of previous methods, as this practice tries to address two difficult and inherently different problems at the same time: identifying true blending pixels inside the trimap region, and estimate accurate alpha values for them. We propose AdaMatting, a new end-to-end matting framework that disentangles this problem into two sub-tasks: trimap adaptation and alpha estimation. Trimap adaptation is a pixelwise classification problem that infers the global structure of the input image by identifying definite foreground, background, and semi-transparent image regions. Alpha estimation is a regression problem that calculates the opacity value of each blended pixel. Our method separately handles these two sub-tasks within a single deep convolutional neural network (CNN). Extensive experiments show that AdaMatting has additional structure awareness and trimap fault-tolerance. Our method achieves the state-ofthe-art performance on Adobe Composition-1k dataset both qualitatively and quantitatively. It is also the current bestperforming method on the alphamatting.com online evaluation for all commonly-used metrics.

Total relighting

“…With significant progress in deep learning, many new matting methods have been proposed that improve upon the state-of-the-art results on classical benchmarks [Rhemann et al 2009]. One such method is that of Cai et al [2019], which we leverage in our work. This work showed the importance of an accurate input trimap, a partitioning of the image into a definite foreground, a definite background, and a boundary area where pixels are an unknown blend of foreground and background colors.…”

“…This work showed the importance of an accurate input trimap, a partitioning of the image into a definite foreground, a definite background, and a boundary area where pixels are an unknown blend of foreground and background colors. As in Cai et al [2019], we disentangle the matting estimation problem into two sub-tasks: trimap refinement and matting estimation, although we add foreground prediction.…”

“…A learned matting module is used to extract a high-quality segmentation mask as well as the color of the foreground from the input portrait. In particular, we employ a deep convolutional network and extend the state-of-the-art method of Cai et al [2019] to predict both alpha matte and foreground color 𝐹 . Details regarding the specific architecture, implementation, and training are provided in Appendix A.…”

“…[Barron and Malik 2015;Debevec et al 2000;Nestmeyer et al 2020;Zhou et al 2019], and in determining alpha mattes and foreground colors, e.g. [Cai et al 2019;Forte and Pitié 2020;Hou and Liu 2019;Lutz et al 2018;Xu et al 2017]. A few techniques simultaneously consider foreground estimation and compositing in a unified framework [Wang and Cohen 2006;Zhang et al 2020b] and produce convincing composites when the input and target lighting conditions are similar.…”

Automatic Trimap Generation by a Multimodal Neural Network

“…Many deep learning-based alpha matting methods have been proposed recently. Some of them require only photos as input [2,3], while others require photos and trimaps [1,4,5,6,7].…”