Autonomous Tracking For Volumetric Video Sequences

Abstract

As a rapidly growing medium, volumetric video is gaining attention beyond academia, reaching industry and creative communities alike. This brings new challenges to reduce the barrier to entry from a technical and economical point of view. We present a system for robustly and autonomously performing temporally coherent tracking for volumetric sequences, specifically targeting those from sparse setups or with noisy output. Our system will detect and recover missing pertinent geometry across highly incoherent sequences as well as provide users the option of propagating drastic topology edits. In this way, affordable multi-view setups can leverage temporal consistency to reduce processing and compression overheads while also generating more aesthetically pleasing volumetric sequences.

Volumetric Video Content Creation for Immersive XR Experiences

“…clothing, number of connected components, etc. Another approach is to perform frame by frame registration [5,6]. This approach is generalizable and can be performed using keyframe strategies similar to that in video compression.…”

“…This approach is generalizable and can be performed using keyframe strategies similar to that in video compression. Moynihan et al [6] propose a system which automatically enforces temporal coherence end-to-end and also allows the user to sculpt the geometry during keyframes which can be tracked across the sequence.…”

“…Figure 4. Mesh tracking and registration is typically applied for efficient representation and to enable editing of VV[6].…”