A stereoscopic video see-through augmented reality system based on real-time vision-based registration

Kanbara, Masayuki; Okuma, Takashi; Takemura, Haruo; Yokoya, Naokazu

doi:10.1109/vr.2000.840506

Cited by 69 publications

(34 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another advantage of HMD-based interfaces is that they have the potential capability of showing correct occlusion phenomena between virtual and real scenes. That is, with a video see-through head mount display and proper depth information of a real scene, virtual objects can cover further real objects, and be covered by closer real objects [4] [5]. However, the video see-through approach degrades the quality of the real scene dramatically and inevitably introduces a system delay.…”

Section: Introductionmentioning

confidence: 99%

An occlusion capable optical see-through head mount display for supporting co-located collaboration

Kiyokawa

Billinghurst²,

Campbell

et al.

The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings.

View full text Add to dashboard Cite

An ideal augmented reality (AR) display for multi-user co-located collaboration should have following three features: 1) Any virtual object should be able to be shown at any arbitrary position, e.g. a user can see a virtual object in front of other users' faces. 2) Correct occlusion of virtual and real objects should be supported.3) The real world should be naturally and clearly visible, which is important for face-to-face conversation. We have been developing an optical see-through display, ELMO (Enhanced see-through display using an LCD panel for Mutual Occlusion), that satisfies these three requirements. While previous prototype systems were not practical due to their size and weight, we have come up with an improved optics design which has reduced size and is lightweight enough to wear. In this paper, the characteristics of typical multi-user three-dimensional displays are summarized and the design details of the latest optics are then described. Finally, a collaborative AR application employing the new display and its user experience are explained.

show abstract

Section: Introductionmentioning

confidence: 99%

An occlusion capable optical see-through head mount display for supporting co-located collaboration

Kiyokawa

Billinghurst²,

Campbell

et al.

The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings.

View full text Add to dashboard Cite

show abstract

“…In that early system, and as previously done in [10,34], real-time registration of the virtual content to the real images was achieved by localizing chromatically distinguishable spherical markers. The video marker-based registration method registers the virtual 3D space to the camera coordinate system (CRS) through real-time determination of the camera pose in the radiological coordinate system (SRS).…”

Section: Introductionmentioning

confidence: 99%

Robust and Accurate Algorithm for Wearable Stereoscopic Augmented Reality with Three Indistinguishable Markers

et al. 2016

View full text Add to dashboard Cite

Abstract:In the context of surgical navigation systems based on augmented reality (AR), the key challenge is to ensure the highest degree of realism in merging computer-generated elements with live views of the surgical scene. This paper presents an algorithm suited for wearable stereoscopic augmented reality video see-through systems for use in a clinical scenario. A video-based tracking solution is proposed that relies on stereo localization of three monochromatic markers rigidly constrained to the scene. A PnP-based optimization step is introduced to refine separately the pose of the two cameras. Video-based tracking methods using monochromatic markers are robust to non-controllable and/or inconsistent lighting conditions. The two-stage camera pose estimation algorithm provides sub-pixel registration accuracy. From a technological and an ergonomic standpoint, the proposed approach represents an effective solution to the implementation of wearable AR-based surgical navigation systems wherever rigid anatomies are involved.

show abstract

“…Most systems can be divided into a capture component and display components. The capture component is usually some kind of image capture device, a single camera [38], a stereo pair [39], or one or more depth sensors [40]. The display component show the user the resulting environment, these can be screens, projectors, head mounted displays (HMD), etc.…”

Section: Introductionmentioning

confidence: 99%

Depth Assisted Composition of Synthetic and Real 3D Scenes

Cortés¹,

Suominen²,

Gotchev³

2016

View full text Add to dashboard Cite

In media production, previsualization is an important step. It allows the director and the production crew to see an estimate of the final product during the filmmaking process. This work focuses on a previsualization system for composite shots which involve real and virtual content. It shows the camera operator a correct perspective view of how the real objects in front of him look placed in a virtual space. The aim is to simplify the workflow, reduce production time and allow more direct control of the end result.The real scene is shot with a 3D scene capture device, which combines an RGB color camera with time-of-flight depth camera. The device's pose is tracked using a motion capture system. Depth-based segmentation is applied to remove the background and content outside the desired volume, the captured geometry is aligned with a stream from the RGB color camera and a dynamic point cloud of the remaining real scene contents is created. The virtual objects are then also transformed into the coordinate space of the tracked camera, and the resulting composite view is rendered accordingly. The prototype camera system is implemented as a self-contained unit with local processing.A prototype system was constructed from a Microsoft Kinect v2, providing depth and color information of the real scene and a Microsoft Surface Pro 3 as a processing and display device. Both instruments were attached to a camera shoulder mount, with optical markers fixed to the body of the camera. The pose of the camera in 3D space is tracked with a Natural Point OptiTrack motion capture system, which streams the location information to the Surface device over a wireless 802.11n channel.At its current state, the system is running at 15 frames per second with a resolution of 1024x768. Subjectively, the frame rate is already smooth enough for the operator to feel as if using a regular camera. Further improvements are targeted in the processing speed and the image quality provided by the system. The image suffers from some depth capture related artefacts which influence the depth segmentation, and therefore adaptive filtering methods based on edge-aware bilateral filtering have been investigated. The tested filtering has improved the quality significantly, while more effort has to be put in implementing the filtering in an efficient way.ii PREFACEThe work done for this thesis was performed as part of the 3D media group in Tampere University of Technology. The work is part of the Centre for Immersive Visual Technology (CIVIT). The aim of the work is to research the potential of new technologies in industry, in particular in media production.

show abstract

A stereoscopic video see-through augmented reality system based on real-time vision-based registration

Cited by 69 publications

References 15 publications

An occlusion capable optical see-through head mount display for supporting co-located collaboration

An occlusion capable optical see-through head mount display for supporting co-located collaboration

Robust and Accurate Algorithm for Wearable Stereoscopic Augmented Reality with Three Indistinguishable Markers

Depth Assisted Composition of Synthetic and Real 3D Scenes

Contact Info

Product

Resources

About