A comparison between two 3D free-viewpoint generation methods: Player-billboard and 3D reconstruction

Shin, Tetsuya; Kasuya, Nozomu; Kitahara, Itaru; Kameda, Yoshinari; Ohta, Yuichi

doi:10.1109/3dtv.2010.5506605

Cited by 11 publications

(9 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…According to subjective experiments in [24], the billboard model approach is more robust to noise than the other visual hull methods, but the main problem in the conventional billboard model method is where the contact point between the billboard and the ground in 3D space is. To be specific, the contact point is calculated using simple projection from the player coordinates in the 2D image to the 3D field, and in the methods [13], [14], the bottom line in the player boundary box is used as a reference for the player coordinates.…”

Section: Related Workmentioning

confidence: 99%

Optimal Billboard Deformation via 3D Voxel for Free-Viewpoint System

Nonaka

Sabirin

Chen

et al. 2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

A free-viewpoint application has been developed that yields an immersive user experience. One of the simple free-viewpoint approaches called "billboard methods" is suitable for displaying a synthesized 3D view in a mobile device, but it suffers from the limitation that a billboard should be positioned in only one position in the world. This fact gives users an unacceptable impression in the case where an object being shot is situated at multiple points. To solve this problem, we propose optimal deformation of the billboard. The deformation is designed as a mapping of grid points in the input billboard silhouette to produce an optimal silhouette from an accurate voxel model of the object. We formulate and solve this procedure as a nonlinear optimization problem based on a grid-point constraint and some a priori information. Our results show that the proposed method generates a synthesized virtual image having a natural appearance and better objective score in terms of the silhouette and structural similarity. key words: free-viewpoint system, billboard model, voxel model, image deformation, convex optimization

show abstract

Section: Related Workmentioning

confidence: 99%

Optimal Billboard Deformation via 3D Voxel for Free-Viewpoint System

Nonaka

Sabirin

Chen

et al. 2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…As the number of multiple cameras increases, the 3D estimation accuracy is also improved as well as the quality of the generated free-viewpoint video. However, SfS estimation accuracy is easily damaged by the noise of the captured images [11].…”

Section: Ibmrmentioning

confidence: 99%

“…In the same way, dolly-in video can also be generated. However, due to such influences as the calibration error of multi-view camera, synchronization shift, changes in lighting conditions, and appearance changes due to sunlight fluctuation, the generated 3D model often includes estimation error [11]. Therefore, [11] has a problem that the quality of the generated dolly-in video may deteriorate.…”

Section: Introductionmentioning

confidence: 99%

Pseudo-Dolly-In Video Generation Combining 3D Modeling and Image Reconstruction

Shishido

Yamanaka²,

Kameda

et al. 2017

2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct)

Self Cite

View full text Add to dashboard Cite

Figure 1: Images of a soccer game. Upper part shows pseudo-dolly video by proposed method. In the pseudo-dolly-in video, motion parallax can be reproduced and the depth can be understood. Furthermore, since the reconstruction process is applied to the missing region, the proposed method maintains the image quality. Lower figure shows digital zoom image. When comparing with digital zoom image and proposed method, in the proposed method, the players are rendered at (relatively) geometrically correct positions as if the camera goes in to the field (dolly-in). ABSTRACTThis paper proposes a pseudo-dolly-in video generation method that reproduces motion parallax by applying image reconstruction processing to multi-view videos. Since dolly-in video is taken by moving a camera forward to reproduce motion parallax, we can present a sense of immersion. However, at a sporting event in a large-scale space, moving a camera is difficult. Our research generates dolly-in video from multi-view images captured by fixed cameras. By applying the Image-Based Modeling technique, dollyin video can be generated. Unfortunately, the video quality is often damaged by the 3D estimation error. On the other hand, Bullet-Time realizes high-quality video observation. However, moving the virtual-viewpoint from the capturing positions is difficult. To solve these problems, we propose a method to generate a pseudo-dollyin image by installing 3D estimation and image reconstruction techniques into Bullet-Time and show its effectiveness by applying it to multi-view videos captured at an actual soccer stadium. In the experiment, we compared the proposed method with digital zoom images and with the dolly-in video generated from the Image-Based Modeling and Rendering method.

show abstract

“…One of the technologies to effectively present a multi-view video is bullet-time video [6,7], which realizes a visual effect of a viewport moving around the subject by sequentially switching/displaying the multi-view videos along with the layout of the shooting cameras. Unlike free-viewpoint video [8][9][10][11][12][13], generating bullet-time video does not require reconstruction of a 3D image of the target scene; thus, it is possible to generate and present a high-quality image with a low processing cost, which is an important issue for visual feedback. Therefore, we consider bullet-time video an appropriate approach for the viewport visual feedback in motor learning.…”

Section: Introductionmentioning

confidence: 99%

An On-site Visual Feedback Method Using Bullet-Time Video

Nagai

Shishido

Kameda

et al. 2018

Proceedings of the 1st International Workshop on Multimedia Content Analysis in Sports

Self Cite

View full text Add to dashboard Cite

This paper describes an on-site visual feedback method that executes all processes from capturing of multi-view videos to generating and displaying bullet-time videos in real-time. In order to realize the on-site visual feedback in a dynamic scene where the subject moves around, such as a sports scene, it is necessary to automatically set the target point to where an observer pays attention. We combine an RGB-D camera that detects the position of the subject with our developed bullet-time video generation method in real-time, and achieve automatic setting of the target point based on the measured 3D position. Furthermore, we incorporate a function to detect a keyframe and automatically switch the viewpoint, to enable easier and more intuitive observation. 1

show abstract

A comparison between two 3D free-viewpoint generation methods: Player-billboard and 3D reconstruction

Cited by 11 publications

References 8 publications

Optimal Billboard Deformation via 3D Voxel for Free-Viewpoint System

Optimal Billboard Deformation via 3D Voxel for Free-Viewpoint System

Pseudo-Dolly-In Video Generation Combining 3D Modeling and Image Reconstruction

An On-site Visual Feedback Method Using Bullet-Time Video

Contact Info

Product

Resources

About