Optimizing multiview video plus depth retargeting technique for stereoscopic 3D displays

Shao, Feng; Lin, Wenchong; Fu, Randi; Yu, Mei; Jiang, Gangyi

doi:10.1364/oe.25.012478

Cited by 9 publications

(4 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ideal view image can be obtained with the ECVIR or BRT methods. Then, based on the average PSNR performance, we compare the outcome of the HRT method with those of state-of-the-art methods, namely, GMMDIBR [20] and MVDRT [22]. We apply different hole-filling methods in GMMDIBR, MVDRT and HRT to refine the blended image.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Emerging holes in a target virtual view can be greatly alleviated by making good use of other neighboring complementary views in addition to the two closest neighboring primary views [21]. Reference [22] presents the multiview video plus depth retargeting (MVDRT) technique for stereoscopic 3D displays, which takes shape preservation, line bending and visual comfort constraints into account and simultaneously optimizes the horizontal, vertical and depth coordinates in the display space. Reference [23] uses color correction of reference views and combines depth-based image fusion with direct color image fusion to decrease ghost effects.…”

Section: B Image-based Rendering Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A Real-Time Super Multiview Rendering Pipeline for Wide Viewing-Angle and High-Resolution 3D Displays Based on a Hybrid Rendering Technique

et al. 2020

View full text Add to dashboard Cite

Super multiview (SMV) 3D displays with a directional diffuser can provide dense viewpoints, wide viewing angles and high image quality. However, traditional SMV 3D rendering pipelines have many disadvantages, such as high time costs, low image quality, incorrect lighting effects and limited viewing angles. To overcome the abovementioned drawbacks, we propose a new SMV 3D rendering pipeline based on a hybrid rendering technique (HRT) that effectively combines and improves the Turing multiview rendering technique, the multiple view plus depth (MVD) technique and the deferred shading technique, introducing additional normal information, diffuseness information and shininess information into the SMV 3D rendering pipeline. The SMV HRT pipeline can generate SMV 3D images with accurate lighting in real time when the viewpoint number of an SMV 3D display is greater than 50, the viewing angle is greater than 100 degrees, the resolution of a single viewpoint image is more than 512 * 512 and the resolution of the LCD panel is 7680 * 4320; in particular, complex scenes can be generated in real time. The proposed HRT pipeline is the only viable real-time rendering pipeline with accurate lighting effects for super multiview 3D displays with wide viewing angle and high resolution that runs on a common PC with standard graphics hardware. An 8k SMV 3D display with an 80 • viewing angle and 100 viewpoints is applied to demonstrate the validity of the HRT method in experiments. When the number of reference views is four, the experimental results show that the rendering frame rate is more than 35 fps on average for complex virtual scenes, and the peak signal-to-noise ratio (PSNR) value of the HRT is greater than 36 dB, illustrating its high image quality and accurate lighting. Compared with state-of-the-art MVD methods, HRT processes more lighting information and is more suitable for virtual scene rendering. Therefore, the HRT method has advantages regarding the image quality of the reconstructed 3D image and can easily achieve real-time rendering for SMV 3D displays with wide viewing angles.INDEX TERMS 3D display, super multiview display, real-time rendering, wide viewing angle, computer-generated 3D image.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: B Image-based Rendering Methodsmentioning

confidence: 99%

A Real-Time Super Multiview Rendering Pipeline for Wide Viewing-Angle and High-Resolution 3D Displays Based on a Hybrid Rendering Technique

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Liu et al [134] warp stereo 3D videos with disparity constraints. And Shao et al [135] attempt to warp multi-view videos with depth.…”

Section: Warpingmentioning

confidence: 99%

AI Video Editing: a Survey

Zhang¹,

Li²,

Han³

et al. 2022

Preprint

View full text Add to dashboard Cite

Video editing is a high-required job, for it requires skilled artists or workers equipped with plentiful physical strength and multidisciplinary knowledge, such as cinematography, aesthetics. Thus gradually, more and more researches focus on proposing semi-automatical and even fully automatical solutions to reduce workloads. Since those conventional methods are usually designed to follow some simple guidelines, they lack flexibility and capability to learn complex ones. Fortunately, the advances of computer vision and machine learning make up the shortages of traditional approaches and make AI editing feasible. There is no survey to conclude those emerging researches yet. This paper summaries the development history of automatic video editing, and especially the applications of AI in partial and full workflows. We emphasizes video editing and discuss related works from multiple aspects: modality, type of input videos, methology, optimization, dataset, and evaluation metric. Besides, we also summarize the progresses in image editing domain, i.e., style transferring, retargeting, and colorization, and seek for the possibility to transfer those techniques to video domain. Finally, we give a brief conclusion about this survey and explore some open problems.

show abstract

“…In order to ensure visual comfort, the regions exceeding the zone of comfort limits should be adjusted. Different with our previous works directly operating on the perceived depth to match such limits [42], in this work, we use a more intuitive scheme that directly maps angular disparities into the (−1 • , 1 • ) limits. As shown in Fig.…”

Section: Constructed Depth Adjustment Modelmentioning

confidence: 99%

Seam Manipulator: Leveraging Pixel Fusion for Depth-Adjustable Stereoscopic Image Retargeting

et al. 2019

Self Cite

View full text Add to dashboard Cite

To enhance the depth perception for stereoscopic image retargeting, this paper presents a depthadjustable stereoscopic image retargeting method via pixel fusion technique. The traditional methods enforce the depth-preserving constraint on the left and right images with poor depth adaption. To solve this issue, we try to address two critical properties that how to change the widths of unmatched region to control the depth perception, and how to change the alignment of left and right seams to satisfy visual comfort limits, and thus propose a seam manipulator tool to optimize the process for scaling factor determination of stereoscopic seams. The important contribution of our method is to take scaling preservation energy and disparity consistency energy into optimization. The experimental results demonstrate that our method can achieve promising performances on depth perception and visual comfort adjustment for better visual experience. INDEX TERMS Stereoscopic image retargeting, pixel fusion, seam carving, visual comfort, depth perception, seam manipulation.

show abstract

Optimizing multiview video plus depth retargeting technique for stereoscopic 3D displays

Cited by 9 publications

References 29 publications

A Real-Time Super Multiview Rendering Pipeline for Wide Viewing-Angle and High-Resolution 3D Displays Based on a Hybrid Rendering Technique

A Real-Time Super Multiview Rendering Pipeline for Wide Viewing-Angle and High-Resolution 3D Displays Based on a Hybrid Rendering Technique

AI Video Editing: a Survey

Seam Manipulator: Leveraging Pixel Fusion for Depth-Adjustable Stereoscopic Image Retargeting

Contact Info

Product

Resources

About