Real time trinocular stereo for tele-immersion

Abstract

Tele-immersion is a technology that augments your space with real-time 3D projections of remote spaces thus facilitating the interaction of people from different places in virtually the same environment. Tele-immersion combines 3D scene recovery from computer vision, and rendering and interaction from computer graphics. We describe the real-time 3D scene acquisition using a new algorithm for trinocular stereo. We extend this method in time by combining motion and stereo in order to increase speed and robustness. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of the University of Pennsylvania's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to pubs-permissions@ieee.org. By choosing to view this document, you agree to all provisions of the copyright laws protecting it.This conference paper is available at ScholarlyCommons: http://repository.upenn.edu/cis_papers/23 REAL TIME TRINOCULAR STEREO FOR TELE-IMMERSION Jane Mulligan and Kostas DaniilidisUniversity of Pennsylvania, GRASP Laboratory 3401 Walnut Street, Philadelphia, PA { janem,kostas} @grip.cis.upenn.edu ABSTRACT Tele-immersion is a technology that augments your space with real-time 3D projections of remote spaces thus facilitating the interaction of people from different places in virtually the same environment. Tele-immersion combines 3D scene recovery from computer vision, and rendering and interaction from computer graphics. We describe the realtime 3D scene acquisition using a new algorithm for trinocular stereo We extend this method in time by combining motion and stereo in order to increase speed and robustness.

Multicamera Real-Time 3D Modeling for Telepresence and Remote Collaboration

“…In contrast, full 3D geometry descriptions allow unconstrained free viewpoints and interactions as they carry more information. They are already used for teleimmersion [2,4,11,12]. Nevertheless, existing 3D human representations, in real-time systems, often have limitations such as imperfect, incomplete, or coarse geometric models, low resolution textures, or slow frame rates.…”

Real-time 2D to 3D video conversion

“…While these may be feasible to implement in real-time on modern high-end computers or dedicated hardware [13][14][15][16][17][18][19][20], they are not suited for conversion of 2D to 3D video on low-end hardware or thin clients. Moreover, while some software solutions for depth map estimation (not including stereo synthesis) for low resolution images (QVGA) and hardware solutions for VGA resolution exist (http://www.…”

Skeleton-Based Data Compression for Multi-camera Tele-Immersion System

“…In the next step, edges of foreground objects are extracted and regions with similar features (i.e. texture) are identified to perform region based correlation among the images captured by the stereo triplet [4]. Next, the depth map is computed from the three gray scale images using triangulation method.…”

“…Different approaches for real-time processing of depth information have been proposed. One of the first TI systems was presented by the researchers at University of Pennsylvania [4]. Their system consisted of several stereo camera triplets used for the image-based reconstruction of the upper body, which allowed a local user to communicate to remote users while sitting behind a desk.…”