Multi-view data capture for dynamic object reconstruction using handheld augmented reality mobiles

An open-source mobile-based system for synchronised multi-view capture and dynamic object reconstruction

“…In industry, our system can promote digital creativity, enabling game development or the creation of social networks based on immersive videos [23]. For example, the results we presented in our prior works [7,8] show that our setup can be used as a 3D motion capture system with consumer mobiles and with a minimal hardware setup. Because our system relies on the SLAM system of Android ARCore [20], which is a continuously evolving Augmented Reality platform maintained by Google, we are confident that the mobile localisation accuracy will also improve with time, thus offering higherquality data captures.…”

“…VideoPose3D [24], our multi-view camera system can benefit from the synchronisation strategy to introduce redundancy in the captured data. In [8], we show how such redundancy is key when the video streams captured from some viewpoints are affected by noise, clutter or occlusions.…”

“…Immersive computing is the next step in human interactions and the potential for immersive digital content is broad, impacting upon entertainment, advertising, gaming, mobile telepresence, and tourism [1][2][3][4][5][6][7][8]. But for immersive computing to become mainstream, systems to easily create large quantities of immersive content are needed.…”

“…In this paper, we present the first open-source mobile-based system to capture nearly-synchronous frame streams from multiple handheld mobiles and a software to reconstruct a dynamic object. A thorough description and assessment of all the modules and algorithms of our system can be found in our previous works [7,8]. To facilitate the use of our reconstruction algorithms we collected an outdoor dataset, namely 4DM (4DMobile), which involves an actor performing sports actions that we recorded using six handheld Augmented Reality mobiles (Fig.…”