Figure 1: We demonstrate the first neural view synthesis method that is optimized to meet the unique requirements for VR passthrough, synthesizing perspective-correct viewpoints in real time and with high visual fidelity. Left: We demonstrate performance using a custom-built VR headset, containing a stereo RGB camera rig with an adjustable baseline. Right: Our method runs in real-time and supports dynamic scenes (top) and near-field objects (bottom).
As virtual reality (VR) devices become increasingly commonplace, asymmetric interactions between people with and without headsets are becoming more frequent. Existing video pass-through VR headsets solve one side of these asymmetric interactions by showing the user a live reconstruction of the outside world. This paper further advocates for
reverse pass-through VR
, wherein a three-dimensional view of the user's face and eyes is presented to any number of outside viewers in a perspective-correct manner using a light field display. Tying together research in social telepresence and copresence, autostereoscopic displays, and facial capture, reverse pass-through VR enables natural eye contact and other important non-verbal cues in a wider range of interaction scenarios, providing a path to potentially increase the utility and social acceptability of VR headsets in shared and public spaces.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.