Optimal Billboard Deformation via 3D Voxel for Free-Viewpoint System

Abstract

A free-viewpoint application has been developed that yields an immersive user experience. One of the simple free-viewpoint approaches called "billboard methods" is suitable for displaying a synthesized 3D view in a mobile device, but it suffers from the limitation that a billboard should be positioned in only one position in the world. This fact gives users an unacceptable impression in the case where an object being shot is situated at multiple points. To solve this problem, we propose optimal deformation of the billboard. The deformation is designed as a mapping of grid points in the input billboard silhouette to produce an optimal silhouette from an accurate voxel model of the object. We formulate and solve this procedure as a nonlinear optimization problem based on a grid-point constraint and some a priori information. Our results show that the proposed method generates a synthesized virtual image having a natural appearance and better objective score in terms of the silhouette and structural similarity. key words: free-viewpoint system, billboard model, voxel model, image deformation, convex optimization

A Robust Billboard-based Free-viewpoint Video Synthesizing Algorithm for Sports Scenes

“…Nevertheless, it still suffers from a narrow baseline. Billboard-based methods [13], [39], [40], [41] construct a single planar billboard for each object in each camera. The billboards rotate around individual points of the virtual stadium as the viewpoint moves, providing walk-through and fly-through experiences.…”