Depth-image-based rendering with spatial and temporal texture synthesis for 3DTV

Abstract

A depth-image-based rendering (DIBR) method with spatial and temporal texture synthesis is presented in this article. Theoretically, the DIBR algorithm can be used to generate arbitrary virtual views of the same scene in a three-dimensional television system. But the disoccluded area, which is occluded in the original views and becomes visible in the virtual views, makes it very difficult to obtain high image quality in the extrapolated views. The proposed view synthesis method combines the temporally stationary scene information extracted from the input video and spatial texture in the current frame to fill the disoccluded areas in the virtual views. Firstly, the current texture image and a stationary scene image, which is extracted from the input video, are warped to the same virtual perspective position by the DIBR method. Then, the two virtual images are merged together to reduce the hole regions and maintain the temporal consistency of these areas. Finally, an oriented exemplar-based inpainting method is utilized to eliminate the remaining holes. Experimental results are shown to demonstrate the performance and advantage of the proposed method compared with other view synthesis methods.

Hole-Filling Algorithm with Spatio-Temporal Background Information for View Synthesis

“…The spatial resolution of "Ballet," "Break Dancers," and "Book Arrival" is 1024 × 768, and that of "Car Park" and "Street" is 1920 × 1088. The proposed hole-filling algorithm was compared to the state-of-the-art spatio-temporal hole-filling algorithms, including those of Xi [17], Koppel [16], and Yao [18]. In addition, the proposed priority function used for the inpainting was compared to other priority functions, including those of Criminisi [13] and Ramirez [22].…”

“…Koppel et al [16] used global motion to estimate temporal information for filling in the disocclusion, which led to geometric distortion caused by copies of the incorrect patches in the inpainting process. Xi et al [17] employed structural similarity to extract BG regions of the depth map between consecutive frames and used temporal BG information to fill in the disocclusion. This method was hampered by the geometric distortions around isolated disocclusions because BG information existing for short periods in the previous frames was not taken into account in the hole-filling.…”

Performance bottleneck analysis and resource optimized distribution method for IoT cloud rendering computing system in cyber-enabled applications

“…All rendering tasks are completed on a cloud rendering server. Although data and development of cloud rendering [36][37][38][39] help the user to solve a lot of personal problems, there is a lack of a unified evaluation criteria for its performance and efficiency. As for the parallel task layer of the IoT cloud rendering computing system, each computing node device has an independent parallel task scheduling module.…”

Non-uniform Depth Compression Control Algorithm for 3D Real Time Video Detection

“…The classical method is using the depth map to calculate parallax, and then through the translation deformations of local image or the view interpolation, forming parallax images [17][18][19][20][21]. The depth information can be calculated from the left and right eye image disparity, and it also can obtain using the depth record camera [22,23].…”