Semantic Human Matting

Chen, Quan; Ge, Tiezheng; Xu, Yanyu; Zhang, Zhiqiang; Yang, Xinxin; Gai, Kun

doi:10.1145/3240508.3240610

Cited by 129 publications

(139 citation statements)

References 19 publications

Supporting

Mentioning

131

Contrasting

Order By: Relevance

“…In [49] the authors presented the first deep image matting approach based on SegNet [2] and significantly outperformed other competitors. Interestingly, this SegNet-based architecture becomes the standard configuration in many recent deep matting methods [3,5,47].…”

Section: Upsampling In Deep Networkmentioning

confidence: 99%

Indices Matter: Learning to Index for Deep Image Matting

Dai

Shen

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

173

228

View full text Add to dashboard Cite

We show that existing upsampling operators can be unified with the notion of the index function. This notion is inspired by an observation in the decoding process of deep image matting where indices-guided unpooling can recover boundary details much better than other upsampling operators such as bilinear interpolation. By looking at the indices as a function of the feature map, we introduce the concept of learning to index, and present a novel index-guided encoder-decoder framework where indices are self-learned adaptively from data and are used to guide the pooling and upsampling operators, without the need of supervision. At the core of this framework is a flexible network module, termed IndexNet, which dynamically predicts indices given an input. Due to its flexibility, IndexNet can be used as a plug-in applying to any off-the-shelf convolutional networks that have coupled downsampling and upsampling stages.We demonstrate the effectiveness of IndexNet on the task of natural image matting where the quality of learned indices can be visually observed from predicted alpha mattes. Results on the Composition-1k matting dataset show that our model built on MobileNetv2 exhibits at least 16.1% improvement over the seminal VGG-16 based deep matting baseline, with less training data and lower model capacity. Code and models has been made available at: https://tinyurl.com/IndexNetV1. * Corresponding author.Figure 1: Alpha mattes of different models. From left to right, Deeplabv3+ [4], RefineNet [30], Deep Matting [49] and Ours.Bilinear upsampling fails to recover subtle details, but unpooling and our learned upsampling operator can produce much clear mattes with good local contrast.

show abstract

Section: Upsampling In Deep Networkmentioning

confidence: 99%

Indices Matter: Learning to Index for Deep Image Matting

Dai

Shen

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

173

228

View full text Add to dashboard Cite

show abstract

“…[3] first introduced the Gestalt laws to the matting problem, making more robust trimap generation possible. More recently, [6,29] utilized neural networks to generate trimaps, greatly improving the matting performances.…”

Section: Related Workmentioning

confidence: 99%

Disentangled Image Matting

Cai¹,

Zhang²,

Fan³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

118

116

View full text Add to dashboard Cite

Most previous image matting methods require a roughlyspecificed trimap as input, and estimate fractional alpha values for all pixels that are in the unknown region of the trimap. In this paper, we argue that directly estimating the alpha matte from a coarse trimap is a major limitation of previous methods, as this practice tries to address two difficult and inherently different problems at the same time: identifying true blending pixels inside the trimap region, and estimate accurate alpha values for them. We propose AdaMatting, a new end-to-end matting framework that disentangles this problem into two sub-tasks: trimap adaptation and alpha estimation. Trimap adaptation is a pixelwise classification problem that infers the global structure of the input image by identifying definite foreground, background, and semi-transparent image regions. Alpha estimation is a regression problem that calculates the opacity value of each blended pixel. Our method separately handles these two sub-tasks within a single deep convolutional neural network (CNN). Extensive experiments show that AdaMatting has additional structure awareness and trimap fault-tolerance. Our method achieves the state-ofthe-art performance on Adobe Composition-1k dataset both qualitatively and quantitatively. It is also the current bestperforming method on the alphamatting.com online evaluation for all commonly-used metrics.

show abstract

“…The alpha matte is eventually computed by the formula provided by Levin et al [3]. Chen et al [6] combine PSPNet [16] with the matting network proposed by Xu et al [7] and simultaneously train them with a fusion module. Zhang et al [17] present a novel matting method taking a single RGB image as input.…”

Section: Image Mattingmentioning

confidence: 99%

“…We also adopt the fusion module proposed in SHM [6]. The final alpha matte combines the alpha values predicted by the matting network and the probability of each class predicted by the trimap network:…”

Section: Matting Networkmentioning

confidence: 99%

“…For example, hair needs a wider boundary than the limbs. Chen et al [6] propose an end-to-end deep learning framework for human matting, which combines a segmentation network with a matting network using a fusion module. Their method can automatically generate alpha mattes, but may miss some body parts or details when the background is complicated.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

JMNet: A joint matting network for automatic human matting

Fang

Chen³

et al. 2020

Comp. Visual Media

View full text Add to dashboard Cite

We propose a novel end-to-end deep learning framework, the Joint Matting Network (JMNet), to automatically generate alpha mattes for human images. We utilize the intrinsic structures of the human body as seen in images by introducing a pose estimation module, which can provide both global structural guidance and a local attention focus for the matting task. Our network model includes a pose network, a trimap network, a matting network, and a shared encoder to extract features for the above three networks. We also append a trimap refinement module and utilize gradient loss to provide a sharper alpha matte. Extensive experiments have shown that our method outperforms state-of-theart human matting techniques; the shared encoder leads to better performance and lower memory costs. Our model can process real images downloaded from the Internet for use in composition applications.

show abstract

Semantic Human Matting

Cited by 129 publications

References 19 publications

Indices Matter: Learning to Index for Deep Image Matting

Indices Matter: Learning to Index for Deep Image Matting

Disentangled Image Matting

JMNet: A joint matting network for automatic human matting

Contact Info

Product

Resources

About