No abstract

No abstract

Conventional approaches to image de-fencing have limited themselves to using only image data in adjacent frames of the captured video of an approximately static scene. In this work, we present a method to harness disparity using a stereo pair of fenced images in order to detect fence pixels. Tourists and amateur photographers commonly carry smartphones/phablets which can be used to capture a short video sequence of the fenced scene. We model the formation of the occluded frames in the captured video. Furthermore, we propose an optimization framework to estimate the de-fenced image using the total variation prior to regularize the ill-posed problem.

Tourists and amateur photographers are often hindered in capturing their cherished images/videos by a fence/occlusion that limits accessibility to the scene of interest. The situation has been exacerbated by growing concerns of security at public places and a need exists to provide a tool that can be used for post-processing such "fenced videos" to produce a "defenced" image. There are several challenges in this problem and in this work, we identify them as 1. Robust detection of the fence/occlusions. 2. Estimating pixel motion of background scene. 3. Filling in the fence/occlusions by utilizing information in multiple frames of the input video. We use a video captured by a camera panning the scene containing a fence and obtain a "de-fenced" image. Our method can effectively remove fences from images as demonstrated for several synthetic and real-world cases.Index Terms-Image de-fencing, inpainting, belief propagation, Markov random field. BACKGROUNDIn recent times, security concerns have led to extra precautions at popular public places and monuments such as fences and barricades. For the tourist, who wishes to capture his memories in an image/video at his favourite landmark, this poses a hindrance which spoils the captured data. It would be so much nice if a post-processing tool existed that can efficiently rid the input video of occlusion artifacts. It is common for the user to pan the camera while capturing a video of the scene in order to cover the entire landscape. A sample frame from a captured video is shown in Fig. 1 (a) wherein the fence is occluding parts of the face and body. We observe that the motion cue in video can be exploited to perform "de-fencing" of the degraded frames to obtain an image wherein the fence has been removed. In Fig. 1 (c), we show a sample output of the proposed algorithm which has successfully removed the occlusions due to fence pixels.There has been considerable progress in the area of image inpainting [3,4,5,6,7] in which most works assume that theFig. 1. Image de-fencing: (a) A frame from the video captured by panning the person occluded by a fence. (b) Estimating the global relative motion of background pixels by matching corresponding points using affine SIFT descriptor [1]. (c) De-fenced image obtained by the proposed algorithm. (d) A result from [2]. (e) Corresponding output of our technique.

Conventiona approaches to image de-fencing use multiple adjacent frames for segmentation of fences in the reference image and are limited to restoring images of static scenes only. In this paper, we propose a de-fencing algorithm for images of dynamic scenes using an occlusionaware optical flow method. We divide the problem of image de-fencing into the tasks of automated fence segmentation from a single image, motion estimation under known occlusions and fusion of data from multiple frames of a captured video of the scene. Specifically, we use a pre-trained convolutional neural network to segment fence pixels from a single image. The knowledge of spatial locations of fences is used to subsequently estimate optical flow in the occluded frames of the video for the final data fusion step. We cast the fence removal problem in an optimization framework by modeling the formation of the degraded observations. The inverse problem is solved using fast iterative shrinkage thresholding algorithm (FISTA). Experimental results show the effectiveness of proposed algorithm.

Key words. Depth-of-field, focus measure, Markov random field, optical microscopy, shape from focus. SummaryShape from focus is an elegant method that estimates the structure of a 3D object from a video of captured frames using the degree of focus as the principal cue. However, the quality of the estimated structure is vulnerable to scene texture. The effect is particularly pronounced for objects that are smooth relative to the magnification of the optical system. In this paper, the shape estimation process is cast as an inverse problem. We exploit spatial dependencies by modeling the shape of the object with a discontinuity-adaptive Markov random field wherein the focus measure profile is used to judiciously control the degree of smoothness. The 3D information is obtained by minimizing a suitably derived energy function that preserves fine details of the underlying structure. We show by experimentation on several real-world specimens that our method yields state-of-the-art performance.

Neural activities recorded using electroencephalography (EEG) are mostly contaminated with eye blink (EB) artifact. This results in undesired activation of brain-computer interface (BCI) systems. Hence, removal of EB artifact is an important issue in EEG signal analysis. Of late, several artifact removal methods have been reported in the literature and they are based on independent component analysis (ICA), thresholding, wavelet transformation, etc. These methods are computationally expensive and result in information loss which makes them unsuitable for online BCI system development. To address the above problems, we have investigated sparsity-based EB artifact removal methods. Two sparsity-based techniques namely morphological component analysis (MCA) and K-SVD-based artifact removal method have been evaluated in our work. MCA-based algorithm exploits the morphological characteristics of EEG and EB using predefined Dirac and discrete cosine transform (DCT) dictionaries. Next, in K-SVD-based algorithm an overcomplete dictionary is learned from the EEG data itself and is designed to model EB characteristics. To substantiate the efficacy of the two algorithms, we have carried out our experiments with both synthetic and real EEG data. We observe that the K-SVD algorithm, which uses a learned dictionary, delivers superior performance for suppressing EB artifacts when compared to MCA technique. Finally, the results of both the techniques are compared with the recent state-of-the-art FORCe method. We demonstrate that the proposed sparsity-based algorithms perform equal to the state-of-the-art technique. It is shown that without using any computationally expensive algorithms, only with the use of over-complete dictionaries the proposed sparsity-based algorithms eliminate EB artifacts accurately from the EEG signals.

No abstract