Inpainting in Omnidirectional Images for Privacy Protection

Abstract

Privacy protection is drawing more attention with the advances in image processing, visual and social media. Photo sharing is a popular activity, which also brings the concern of regulating permissions associated with shared content. This paper presents a method for protecting user privacy in omnidirectional media, by removing parts of the content selected by the user, in a reversible manner. Object removal is carried out using three different state-of-the-art inpainting methods, employed over the mask drawn in the viewport domain so that the geometric distortions are minimized. The perceived quality of the scene is assessed via subjective tests, comparing the proposed method against inpainting employed directly on the equirectangular image. Results on distinct contents indicate our object removal methodology on the viewport enhances perceived quality, thereby improves privacy protection as the user is able to hide objects with less distortion in the overall image.

Audio–visual object removal in 360-degree videos

“…The equirectangular projection is the most used; however, there are two difficulties when one applies typical image/video processing techniques to the equirectangular format. First, it inevitably results in severe distortion [23,27]. Second, stepping over the boundaries of an equirectangular projection may lead to unexpected output [29].…”

“…Image inpainting is a technique for filling a "hole" on a target image with a plausible and natural appearance, and it can be used for object removal for images. Many methods have been proposed, including recent learning-based [1,11,23,28], partial differential equation (PDE)-based [3], exemplar-based [13], and wavelet transform-based [2] methods. However, the distortion caused by equirectangular projection is prone to poor results when directly applying these methods proposed for flat images.…”

“…Removing specific targets from an image is a well studied topic within the field of computer vision and computer graphics. Image inpainting can be used to remove a specified target object visually from an original image and replace it with the appearance of something else that fits the image (e.g., background) [1,2,11,23,28]. Recently, several methods have been proposed that are capable of handling video clips rather than just still images [25,26].…”

“…Recently, several methods have been proposed that are capable of handling video clips rather than just still images [25,26]. Nevertheless, to our knowledge, only few papers [17,23] have discussed 360-degree videos, and, more importantly, none of these methods take auditory information into consideration. It is crucial that the visual and auditory information remain well-synchronized even after the object removal process; otherwise, a disparity between the visual and auditory domains could provide unnatural experiences.…”

High dynamic range image sharing with privacy protection

“…This editing methods are: (i) adding stickers, (ii) adding bubbles, (iii) processing filters, (iv) add frames, (v) pixelation and (vi) inpainting. 10 Once editing process is finished, users choose the group of users allowed to view the private image, Mask Matrix is generated by comparing private and public images and computed mask and both images are uploaded to the server where JPEG transmorphing is performed.…”