A Smartphone-Based 3d Pipeline for the Creative Industry – The Replicate Eu Project

Abstract

During the last two decades we have witnessed great improvements in ICT hardware and software technologies. Three-dimensional content is starting to become commonplace now in many applications. Although for many years 3D technologies have been used in the generation of assets by researchers and experts, nowadays these tools are starting to become commercially available to every citizen. This is especially the case for smartphones, that are powerful enough and sufficiently widespread to perform a huge variety of activities (e.g. paying, calling, communication, photography, navigation, localization, etc.), including just very recently the possibility of running 3D reconstruction pipelines. The REPLICATE project is tackling this particular issue, and it has an ambitious vision to enable ubiquitous 3D creativity via the development of tools for mobile 3D-assets generation on smartphones/tablets. This article presents the REPLICATE project’s concept and some of the ongoing activities, with particular attention being paid to advances made in the first year of work. Thus the article focuses on the system architecture definition, selection of optimal frames for 3D cloud reconstruction, automated generation of sparse and dense point clouds, mesh modelling techniques and post-processing actions. Experiments so far were concentrated on indoor objects and some simple heritage artefacts, however, in the long term we will be targeting a larger variety of scenarios and communities.

Towards Gesture-Based Multi-User Interactions In Collaborative Virtual Environments

“…Cardboard devices communicate through WiFi thus facilitating users to collaborate in the same VR environment without necessary being in the same physical place. We have showed the usefulness of our system through a forensic use-case, where objects of a simulated crime scene were reconstructed using a smartphone-based reconstruction pipeline (Nocerino et al, 2017) and then included in the VR environment.…”

“…We use a smartphone-based 3D reconstruction pipeline to scan real-world objects and create their 3D models that we can import into a virtual environment (Nocerino et al, 2017). A smartphone App acquires images and uploads them to a reconstruction server that progressively creates 3D models using algorithms such as Structure from Motion (Wu, 2013), Multi-View Stereo (Locher et al, 2016) and Meshing (Kazhdan et al, 2006) (Schonberger and Frahm, 2016).…”

Open Source and Independent Methods for Bundle Adjustment Assessment in Close-Range UAV Photogrammetry

“…The programs listed above perform the computations locally. Other software programs that perform the computations in the cloud include Autodesk's Recap, KU Leuven's Arc3D [39] and the Replicate project [10].…”

“…Developments in imaging sensor technology have made this method a great alternative or complement to terrestrial laser scanners [7][8][9]. Furthermore, the use of low-cost sensors such as smartphone images has also increased [10].…”

Potential of remote sensing and open street data for flood mapping in poorly gauged areas: a case study in Gonaives, Haiti

“…In the last years, a great number of works in the field of Geomatics have focused their attention on the use of low-cost and commercial sensors for the rapid mapping in different scenarios (Kolev et al 2014;Micheletti et al 2014;Nocerino et al 2017). In Italy, due to the recent seismic events, the attention was focused also in post disaster scenarios, were the contribution of geomatics could become imperative.…”