User-perspective augmented reality magic lens from gradients

Abstract

In this paper we present a new approach to creating a geometricallycorrect user-perspective magic lens and a prototype device implementing the approach. Our prototype uses just standard color cameras, with no active depth sensing. We achieve this by pairing a recent gradient domain image-based rendering method with a novel semi-dense stereo matching algorithm inspired by PatchMatch. Our stereo algorithm is simple but fast and accurate within its search area. The resulting system is a real-time magic lens that displays the correct user perspective with a high-quality rendering, despite the lack of a dense disparity map.

User-Perspective AR Magic Lens from Gradient-Based IBR and Semi-Dense Stereo

“…Although our prototype system is tethered to a workstation and powered by a GPU, we are confident that given the rate of advancement of mobile hardware this will be possible on a self-contained mobile platform in just a few years. This paper extends our previous work [1] in several ways. The stereo pipeline has been substantially reworked, and the algorithm modified to make use of adaptive support weights.…”

“…In this paper we extend our previous work [1] and present a new approach to solving the problem of creating a user-perspective magic lens. We observe that accurate dense scene reconstruction is a requirement imposed by the traditional rendering methods, and not an inherent requirement of creating a user-perspective view.…”

Virtual annotations of the surgical field through an augmented reality transparent display

“…In addition to addressing the low-level limitations as described above, we will work towards improving the transparent display approximation provided by the system, building upon prior work into simulated transparent displays [4,25,27]. The current system does not achieve perfect visual continuity between the parts of the surgical field seen through the display and the parts seen directly (Fig.…”

“…The advantage of passive stereo acquisition is robustness with strong environment illumination, such as in the case of outdoor scenes. The classic disadvantage is the difficulty in establishing correspondences for scenes with little color variation [4]. Unuma et al created a system that relies on active depth acquisition, which improves density, rate, and robustness.…”

Distortion Calibration and FOV Adjustment in Video See-through AR using Mobile Phones

“…이러한 투명유사 디스플레이에 대한 연구들은 사용자 시점 렌더링(user perspective rendering)이라고 불리며 이루 어져 왔다. 이 중에서는 gradient를 이용한 연구나 [6] 카메라 영상의 호모그래피 변환 [7] 을 이용한 연구들이 있다. .…”