An end-to-end system for free viewpoint video for smooth camera transitions

Abstract

In this paper, we present an end-to-end system for free viewpoint video for smooth camera transitions in sport scenes. Our system consists of a network of static computer vision cameras, a storage infrastructure and an interpolation rendering module, connected with a 10 Gigabit Ethernet network. The user of the system requests a viewpath for the virtual camera and the rendering module then generates the images using a depth-aware plane sweep approach. First, the foreground and background are separated and rendered independently. The foreground is rendered using a plane-sweep approach and the obtained depth map is split up in groups of players. Each group is assigned a global depth, which is used in a second plane sweep to restrict the depth. This will reduce artifacts, such as extra limbs and ghost players. The algorithm is demonstrated on actual soccer recordings. The system is fully automatic and can work in near real-time, thus providing virtual images of high quality in a fast manner.

Free Viewpoint Video for Soccer using Histogram-based Validity Maps in Plane Sweeping

“…Plane sweeping has already been employed for interpolation in soccer scenes. Goorts et al (Goorts et al, 2012a;Goorts et al, 2013a) present a method with two plane sweeps and a depth filtering step, but segments in the virtual image can only have one depth. This will result in disappearing players if they are overlapping in the image.…”

“…The quarter arc setup proved to be especially useful for showing frozen action shots from different angles. Other setups with a smaller baseline or a linear camera placement are also possible (Goorts et al, 2012a). However, we demonstrate a method where the distance between the cameras is increased, allowing a more flexible and less expensive setup, thus creating a more practical solution.…”

Fast View Synthesis for Immersive Video Systems

“…Such kind of systems allow a viewer to immerse into a scene, i.e. to virtually navigate within a scene that was captured by a set of arbitrarily located cameras [Goo12][Sta18] [Zit04] ( Fig. 1).…”

Optimization of Free Viewpoint Interpolation by Applying Adaptive Depth Plane Distributions in Plane Sweeping - A Histogram-based Approach to a Non-uniform Plane Distribution

“…Using view interpolation, it is possible to generate images from a nonexisting camera views using the images from the real cameras. This can increase the user experience, for example in sports broadcasting (Goorts et al, 2012a;Goorts et al, 2013) and video conferencing ).…”