Enabling multiparty 3D tele-immersive environments with ViewCast

Abstract

Three-dimensional tele-immersive (3DTI) environments have great potential to promote collaborative work among geographically distributed users. However, most existing 3DTI systems work with only two sites due to the huge demand of resources and the lack of a simple yet powerful networking model to handle connectivity, scalability, and quality-of-service (QoS) guarantees. In this paper, we explore the design space from the angle of multi-stream management to enable multi-party 3DTI communication. Multiple correlated 3D video streams are employed to provide a comprehensive representation of the physical scene in each 3DTI environment, and rendered together to establish a common cyberspace among all participating 3DTI environments. The existence of multi-stream correlation provides the unique opportunity for new approaches in QoS provisioning. Previous work mostly concentrated on compression and adaptation techniques on the per stream basis while ignoring the application layer semantics and the coordination required among streams. We propose an innovative and generalized ViewCast model to coordinate the multi-stream content dissemination over an overlay network. ViewCast leverages view semantics in 3D free-viewpoint video systems to fill the gap between the high-level user interest and the low-level stream management. In ViewCast, only the view information is specified by the user/application, while the underlying control dynamically performs stream differentiation, selection, coordination and dissemination. We present the details of ViewCast and evaluate it through both simulation and 3DTI sessions among tele-immersive environments residing in different institutes across the Internet2. Our experimental results demonstrate the implementation feasibility and performance enhancement of ViewCast in supporting the multi-party 3DTI collaboration.

Integration of Multisensorial Stimuli and Multimodal Interaction in a Hybrid 3DTV System

“…This has been exploited during the last decade thanks to the recent development of 3D technologies and research in the capture, production, and delivery of stereoscopic content. Some works have proposed the use of 3D environments to offer immersion in tele-presence applications, such as multi-party communications [Yang et al 2010]. Traditionally, hearing has been enhanced by means of multi-channel surround audio monitors located around the listener.…”

Hincent: Quick content distribution with priorities and high incentives

“…They do not give a mechanism to prioritize content transfers which is an important component of 3D [10] and other streaming applications. Unlike Hincent they also do not provide an efficient max/min rate allocation mechanism.…”

“…Besides, such existing schemes do not find and enforce accurate rates at which peers can download content from other peers so as to minimize content transfer time. They do not give a mechanism to prioritize content transfers which is an important component of 3D [10] and other multi-view streaming applications where some streams are more important than others based on the view angle. Besides, existing work does not provide an efficient content source selection mechanism which chooses a source that leads to high throughput and low file completion time.…”

QoS Adaptation for Realizing Interaction Between Virtual and Real Worlds Through Wireless LAN

“…TEEVE (Yang et al (2010)) displays 3D live visuals from each user's view in real-time, and constructs an environment for cooperative working. However, expensive and specialized devices and infrastructures such as 3D cameras and broadband network environments are needed to construct tele-immersion environment.…”

“…In TEEVE (Yang et al (2010)), images captured by 3D multi-cameras are processed and transferred through Internet2. Since we are aiming at inexpensively implementing the functions, we decided to measure AR information using inexpensive sensors.…”