Fast Plane-Based Free-viewpoint Synthesis for Real-time Live Streaming

Nonaka, Kenichi; Watanabe, Ryosuke; Sabirin, Houari; Naito, Seiji

doi:10.1109/vcip.2018.8698648

Cited by 13 publications

(17 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, Matsuyama et al 8) achieve real-time FVV generation using multiple planes-based concept with a computer cluster. After that, Nonaka et al 9) have extended this concept to improve the quality of FVV by using view-dependent multiple planes. These methods achieve real-time processing including the silhouette extraction process.…”

Section: Related Work 1 Overview Of Model-based Free Viewpointmentioning

confidence: 99%

“…In a conventional real-time FVV method using multiple planes 9) , an object silhouette is extracted by a background subtraction method using single Gaussian distribution 34) . The method succeeds in extracting a silhouette for 16 Full-HD images in real-time.…”

Section: Silhouette Extraction Technologymentioning

confidence: 99%

“…Next, an object silhouette is extracted by a conventional background subtraction method 34) using a single Gaussian. As with the previous real-time FVV study 9) , a background model is constructed as single Gaussian distribution which is composed of the mean μ c (u, v) and the standard deviation σ c (u, v). The mean and the standard deviation are calculated by the same process as the conventional method 34) After acquiring Gaussian distribution, a silhouette is extracted by the background subtraction method between a camera image and the background model constructed by the single Gaussian distribution.…”

Section: Silhouette Extractionmentioning

confidence: 99%

See 2 more Smart Citations

[Paper] Quality Improvement for Real-time Free Viewpoint Video Using View-dependent Shape Refinement

Watanabe

Konno

Nonaka

et al. 2021

MTA

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a quality improvement method of a real-time free viewpoint video (FVV) synthesis.Two functions, adaptive silhouette dilation and view-dependent shape refinement, are applied to obtain a 3D model with accurate shapes in the real-time FVV. The adaptive dilation reduces the missing part of a reconstructed 3D model caused by a camera calibration error. In addition, the excessively expanded part of a 3D model caused by the silhouette dilation is corrected by the view-dependent refinement algorithm on a screen rendering. A part of the model outlines becomes transparently displayed to refine the model shape by comparing the camera image used for texture mapping with a background model.In the experiments, the proposed method achieved the best quality compared with conventional real-time FVV methods. Furthermore, the processing was executed in real-time because the computation cost of the proposed functions was extremely small.

show abstract

Section: Related Work 1 Overview Of Model-based Free Viewpointmentioning

confidence: 99%

Section: Silhouette Extraction Technologymentioning

confidence: 99%

Section: Silhouette Extractionmentioning

confidence: 99%

See 1 more Smart Citation

[Paper] Quality Improvement for Real-time Free Viewpoint Video Using View-dependent Shape Refinement

Watanabe

Konno

Nonaka

et al. 2021

MTA

Self Cite

View full text Add to dashboard Cite

show abstract

“…The real-time virtual view synthesis methods are also known, but they usually require dedicated FPGA [Aki15][Li19] [Wan12] or VLSI [Hua19] devices or powerful graphic cards [Non18][Yao16] [Zha17].…”

Section: Introductionmentioning

confidence: 99%

Fast View Synthesis for Immersive Video Systems

Stankowski¹,

Dziembowski²

2020

Computer Science Research Notes

View full text Add to dashboard Cite

Immersive video has become a popular research topic recently. However, there are no fast immersive video processing methods, which could be used in practical immersive video systems. In this paper the real-time CPUbased virtual view synthesis method is presented. The proposed method allows a viewer to freely navigate within acquired scene without necessity of using dedicated FPGA devices or powerful graphic cards. Presented view synthesis method can be used in practical immersive video systems, even for ultra-high resolution sequences. In order to present usefulness of proposed method, several implementations and use cases are discussed in the paper.

show abstract

“…Compared with traditional fixed-viewpoint video, it allows users to select a viewpoint interactively and is capable of rendering a new view from a novel viewpoint. Since a virtualized reality system [1] that distributes 51 cameras over a 5 m dome with controlled lighting and wellcalibrated cameras was introduced, FVV has been a longstanding research topic in the field of computer vision ranging from model construction of a static object for films [2] to the generation of dynamic object models for sports scenes [3]- [5]. Moreover, this has not been confined to academia, the companies LiberoVision, Intel, and 4DViews also attach importance to the technique and have been providing visual effects applications for various purposes.…”

Section: Introductionmentioning

confidence: 99%

Fast Free-viewpoint Video Synthesis Algorithm for Sports Scenes

Watanabe

Nonaka

Konno

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

In this paper, we report on a parallel freeviewpoint video synthesis algorithm that can efficiently reconstruct a high-quality 3D scene representation of sports scenes. The proposed method focuses on a scene that is captured by multiple synchronized cameras featuring wide-baselines. The following strategies are introduced to accelerate the production of a free-viewpoint video taking the improvement of visual quality into account: (1) a sparse point cloud is reconstructed using a volumetric visual hull approach, and an exact 3D ROI is found for each object using an efficient connected components labeling algorithm. Next, the reconstruction of a dense point cloud is accelerated by implementing visual hull only in the ROIs; (2) an accurate polyhedral surface mesh is built by estimating the exact intersections between grid cells and the visual hull; (3) the appearance of the reconstructed presentation is reproduced in a view-dependent manner that respectively renders the non-occluded and occluded region with the nearest camera and its neighboring cameras. The production for volleyball and judo sequences demonstrates the effectiveness of our method in terms of both execution time and visual quality.

show abstract

Fast Plane-Based Free-viewpoint Synthesis for Real-time Live Streaming

Cited by 13 publications

References 7 publications

[Paper] Quality Improvement for Real-time Free Viewpoint Video Using View-dependent Shape Refinement

[Paper] Quality Improvement for Real-time Free Viewpoint Video Using View-dependent Shape Refinement

Fast View Synthesis for Immersive Video Systems

Fast Free-viewpoint Video Synthesis Algorithm for Sports Scenes

Contact Info

Product

Resources

About