A multimodal approach for image de-fencing and depth inpainting

Jonna, Sankaraganesh; Voleti, Vikram; Sahay, Rajiv R.; Kankanhalli, Mohan

doi:10.1109/icapr.2015.7050696

Cited by 19 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work, we propose a depth-based technique to detect the fence mask from the disparity map obtained from a stereo image pair. The works of Wang et al [17] and Jonna et al [18] are most closely related to our method. The algorithm in [17] leveraged advantages of stereo image pair for joint completion of missing texture and depth.…”

Section: Related Workmentioning

confidence: 90%

“…The limitation of [17] is that the region to be inpainted has to be specified manually. The technique of [18] proposed a multimodal approach for image de-fencing where the fence masks are extracted automatically from depth maps corresponding to the color images obtained using Kinect sensor. One limitation of the Kinect sensor is that it provides the depth map of a scene in a constrained laboratory environment and only works accurately if the scene is within a particular distance from sensor (viewing distance range is 1.2m to 3.5m).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Stereo image de-fencing using smartphones

Jonna

Satapathy

Sahay

2017

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Conventional approaches to image de-fencing have limited themselves to using only image data in adjacent frames of the captured video of an approximately static scene. In this work, we present a method to harness disparity using a stereo pair of fenced images in order to detect fence pixels. Tourists and amateur photographers commonly carry smartphones/phablets which can be used to capture a short video sequence of the fenced scene. We model the formation of the occluded frames in the captured video. Furthermore, we propose an optimization framework to estimate the de-fenced image using the total variation prior to regularize the ill-posed problem.

show abstract

Section: Related Workmentioning

confidence: 90%

Section: Related Workmentioning

confidence: 99%

Stereo image de-fencing using smartphones

Jonna

Satapathy

Sahay

2017

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Their proposed algorithms achieve great performance in video-based de-fencing. In addition to RGB video-based methods, method [10] incorporates depth map to enhance the estimated fence mask.…”

Section: Related Work a Video-based Fence Removalmentioning

confidence: 99%

Single-Image Fence Removal Using Deep Convolutional Neural Network

Matsui

Ikehara

2020

IEEE Access

View full text Add to dashboard Cite

In public spaces such as zoos and sports facilities, the presence of fences often annoys tourists and professional photographers. There is a demand for a post-processing tool to produce a non-occluded view from an image or video. This ''de-fencing'' task is divided into two stages: one to detect fence regions and the other to fill the missing part. For over a decade, various methods have been proposed for video-based de-fencing. However, only a few single-image-based methods are proposed. In this paper, we focus on single-image fence removal. Conventional approaches suffer from inaccurate and non-robust fence detection and inpainting due to less content information. To solve these problems, we combine novel methods based on a deep convolutional neural network (CNN) and classical domain knowledge in image processing. In the training process, we are required to obtain both fence images and corresponding non-fence ground truth images. Therefore, we synthesize natural fence images from real images. Moreover, spacial filtering processing (e.g. a Laplacian filter and a Gaussian filter) improves the performance of the CNN for detection and inpainting. Our proposed method can automatically detect a fence and generate a clean image without any user input. Experimental results demonstrate that our method is effective for a broad range of fence images.INDEX TERMS De-fencing, deep learning, image restoration, object removal, convolutional neural network.

show abstract

“…This method uses an image matting (Zheng and Kambhamettu [2009]) for fence segmentation, but the main drawback of this approach is that it involves significant user interaction and is therefore not very suitable for practical purposes. Jonna et al [2015b] proposed a multimodal approach for image de-fencing in video frames, in which the fence mask was first extracted in each frame with the aid of depth maps corresponding to the color images obtained from a Kinect sensor, and next an optical flow algorithm was used to find correspondences between adjacent frames. Finally, estimation of the de-fenced image was done by modeling it as a Markov Random Field, and obtaining its maximum a-posteriori estimate by applying loopy belief propagation.…”

Section: Image De-fencingmentioning

confidence: 99%

Fully Automated Image De-fencing using Conditional Generative Adversarial Networks

Gupta¹,

Jain²,

Tripathi³

et al. 2019

Preprint

View full text Add to dashboard Cite

Image de-fencing is one of the important aspects of recreational photography in which the objective is to remove the fence texture present in an image and generate an aesthetically pleasing version of the same image without the fence texture. In this paper, we aim to develop an automated and effective technique for fence removal and image reconstruction using conditional Generative Adversarial Networks (cGANs). These networks have been successfully applied in several domains of Computer Vision focusing on image generation and rendering. Our initial approach is based on a two-stage architecture involving two cGANs that generate the fence mask and the inpainted image, respectively. Training of these networks is carried out independently and, during evaluation, the input image is passed through the two generators in succession to obtain the de-fenced image. The results obtained from this approach are satisfactory, but the response time is long since the image has to pass through two sets of convolution layers. To reduce the response time, we propose a second approach involving only a single cGAN architecture that is trained using the ground-truth of fenced de-fenced image pairs along with the edge map of the fenced image produced by the Canny Filter. Incorporation of the edge map helps the network to precisely detect the edges present in the input image, and also imparts it an ability to carry out high quality de-fencing in an efficient manner, even in the presence of a fewer number of layers as compared to the two-stage network. Qualitative and quantitative experimental results reported in the manuscript reveal that the de-fenced images generated by the single-stage de-fencing network have similar visual quality to those produced by the two-stage network. Comparative performance analysis also emphasizes the effectiveness of our approach over state-of-the-art image de-fencing techniques.Preprint. Under review.

show abstract

A multimodal approach for image de-fencing and depth inpainting

Cited by 19 publications

References 20 publications

Stereo image de-fencing using smartphones

Stereo image de-fencing using smartphones

Single-Image Fence Removal Using Deep Convolutional Neural Network

Fully Automated Image De-fencing using Conditional Generative Adversarial Networks

Contact Info

Product

Resources

About