A survey of diminished reality: Techniques for visually concealing, eliminating, and seeing through real objects

Mori, Shohei; Ikeda, Sei; Saitō, Hideo

doi:10.1186/s41074-017-0028-1

Cited by 114 publications

(47 citation statements)

References 103 publications

(289 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Virtual reality (VR) and augmented reality (AR) are currently major topics in the media researches to create new immersive experiences [4,30]. Spatial audio is important for the sense of immersion in a virtual or augmented space but previous research has primarily focused more on visual experience.…”

Section: Introductionmentioning

confidence: 99%

Immersive Spatial Audio Reproduction for VR/AR Using Room Acoustic Modelling from 360° Images

Kim

Hernaggi

Jackson

et al. 2019

2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)

View full text Add to dashboard Cite

Recent progresses in Virtual Reality (VR) and Augmented Reality (AR) allow us to experience various VR/AR applications in our daily life. In order to maximise the immersiveness of user in VR/AR environments, a plausible spatial audio reproduction synchronised with visual information is essential. In this paper, we propose a simple and efficient system to estimate room acoustic for plausible reproducton of spatial audio using 360 • cameras for VR/AR applications. A pair of 360 • images is used for room geometry and acoustic property estimation. A simplified 3D geometric model of the scene is estimated by depth estimation from captured images and semantic labelling using a convolutional neural network (CNN). The real environment acoustics are characterised by frequency-dependent acoustic predictions of the scene. Spatially synchronised audio is reproduced based on the estimated geometric and acoustic properties in the scene. The reconstructed scenes are rendered with synthesised spatial audio as VR/AR content. The results of estimated room geometry and simulated spatial audio are evaluated against the actual measurements and audio calculated from ground-truth Room Impulse Responses (RIRs) recorded in the rooms.

show abstract

Section: Introductionmentioning

confidence: 99%

Immersive Spatial Audio Reproduction for VR/AR Using Room Acoustic Modelling from 360° Images

Kim

Hernaggi

Jackson

et al. 2019

2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)

View full text Add to dashboard Cite

show abstract

“…For instance, it can be exploited by robots to autonomously carry out tasks [24]. Virtual reality (VR) and augmented reality (AR) are currently two major topics in the audio-visual research [5,7,36]. Several VR software development kits (SDKs) are freely available to be used by researchers.…”

Section: Introductionmentioning

confidence: 99%

An Audio-Visual Method for Room Boundary Estimation and Material Recognition

Remaggi

Kim

Jackson

et al. 2018

Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia

View full text Add to dashboard Cite

In applications such as virtual and augmented reality, a plausible and coherent audiovisual reproduction can be achieved by deeply understanding the reference scene acoustics. This requires knowledge of the scene geometry and related materials. In this paper, we present an audiovisual approach for acoustic scene understanding. We propose a novel material recognition algorithm, that exploits information carried by acoustic signals. The acoustic absorption coefficients are selected as features. The training dataset was constructed by combining information available in the literature, and additional labeled data that we recorded in a small room having short reverberation time (RT60). Classic machine learning methods are used to validate the model, by employing data recorded in five rooms, having different sizes and RT60s. The estimated materials are utilized to label room boundaries, reconstructed by a visionbased method. Results show 89 % and 80 % agreement between the estimated and reference room volumes and materials, respectively.

show abstract

“…Contrary to Augmented Reality, Diminished Reality (DR) [1] aims at virtually removing one or more objects from the image of a scene. The image regions containing the object must then be filled seamlessly with a texture coherent with the surrounding regions in order to provide a diminished, photo-realistic view.…”

Section: Introductionmentioning

confidence: 99%

Radiometric confidence criterion for patch-based inpainting

Fayer¹,

Morin²,

Gasparini³

et al. 2018

2018 24th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

Diminished Reality (DR) consists in virtually removing objects from a captured scene, thus requiring a coherent filling of the areas originally hidden behind these objects. Indoor DR applications often exploit the planar geometry of the scene to apply an inpainting process on a perspectively undistorted view of the plane. In this paper we propose to integrate a novel, physic-based criterion into classical state-of-the-art inpainting algorithms in order to take into account the variations in image resolution of the undistorted view. The proposed inpainting process selects the patches and avoids the propagation of lowresolution data, i.e. patches corresponding to parts of the plane that are far from the camera or seen under an very skew angle. We illustrate the improvements of DR results on synthetic and real images.

show abstract

A survey of diminished reality: Techniques for visually concealing, eliminating, and seeing through real objects

Cited by 114 publications

References 103 publications

Immersive Spatial Audio Reproduction for VR/AR Using Room Acoustic Modelling from 360° Images

Immersive Spatial Audio Reproduction for VR/AR Using Room Acoustic Modelling from 360° Images

An Audio-Visual Method for Room Boundary Estimation and Material Recognition

Radiometric confidence criterion for patch-based inpainting

Contact Info

Product

Resources

About