Immersive 3D Holoscopic Video System

Abstract

R ecent film releases such as Avatar have revolutionized cinema by combining 3D technology and content production and real actors, leading to the creation of a new genre at the outset of the 2010s. The success of 3D cinema has led several major consumer electronics manufacturers to launch 3D-capable televisions and broadcasters to offer 3D content. Today's 3DTV technology is based on stereo vision, which presents left-and right-eye images through temporal or spatial multiplexing to viewers wearing a pair of glasses. The next step in 3DTV development will likely be a multiview autostereoscopic imaging system, which will record and present many pairs of video signals on a display and will not require viewers to wear glasses. 1,2 Although researchers have proposed several autostereoscopic displays, the resolution and viewing position is still limited. Furthermore, stereo and multiview technologies rely on the brain to fuse the two disparate images to create the 3D effect. As a result, such systems tend to cause eye strain, fatigue, and headaches after prolonged viewing because users are required to focus on the screen plane (accommodation) but to converge their eyes to a point in space in a different plane (convergence), producing unnatural viewing. Recent advances in digital technology have eliminated some of these human factors, but some intrinsic eye fatigue will always exist with stereoscopic 3D technology. 3 These facts have motivated researchers to seek alternative means for capturing true 3D content, most notably holography and holoscopic imaging. Due to the interference of the coherent light fields required to record holograms, their use is still limited and mostly confined to research laboratories. Holoscopic imaging (also referred to as integral imaging) in its simplest form, on the other hand, consists of a lens array mated to a digital sensor with each lens capturing perspective views of the scene. 49 In this case, the light field does not need to be coherent, so holoscopic color images can be obtained with full parallax. This conveniently lets us adopt more conventional live capture and display procedures. Furthermore, 3D holoscopic imaging offers fatigue-free viewing to more than one person, independent of the viewers' positions.Due to recent advances in theory and microlens manufacturing, 3D holoscopic imaging is becoming a practical, prospective 3D display technology and is thus attracting much interest in the 3D area. The 3D Live Immerse VideoAudio Interactive Multimedia (3D Vivant, www.3dvivant.eu) project, funded by the EU-FP7 ICT-4-1.5Networked Media and 3D Internet, has proposed advances in 3D holoscopic imaging technology for the capture, representation, processing, and display of 3D holoscopic content that overcome most of the aforementioned restrictions faced by traditional 3D technologies. This article presents our work as part of the 3D Vivant project. 3D Holoscopic Content GenerationThe 3D holoscopic imaging technique creates and represents a true volume spatial optical model of the objec...

HEVC-based 3D holoscopic video coding using self-similarity compensated prediction

“…Briefly, the holoscopic imaging system comprises a regularly spaced array of small microlenses, known as a "fly's eye" lens array [1], which is used for both acquisition and display of the 3D holoscopic images, as can be seen in Fig. 1.…”

“…Although the public acceptance of the current 3D stereo technology is far from what the industry expected, the research on alternative 3D acquisition and display systems has been rapidly progressing [1][2][3].…”

Dynamic Hyperlinker: Innovative Solution for 3D Video Content Search and Retrieval

“…The dynamic hyperlinker is an advanced holoscopic [19] 3D video player that has hyperlinking capability to ease the 3D search and retrieval. It links objects in the scene to similar/matching object(s) in the repository, while the video clip is being played.…”

Joint processing and fast encoding algorithm for multi-view depth video

“…As computing, communication, and multimedia technologies rapidly advance, users are increasingly interested in three-dimensional (3D) video system applications, such as 3D television, free-viewpoint television, and photorealistic rendering of 3D scenes [1,2]. Multi-view video plus depth (MVD) is a mainstream format that represents 3D scenes.…”