Towards Digital Twins of 3D Reconstructed Apparel Models with an End-to-End Mobile Visualization

Abstract

Digital twin technologies are still developing and are being increasingly leveraged to facilitate daily life activities. This study presents a novel approach for leveraging the capability of mobile devices for photo collection, cloud processing, and deep learning-based 3D generation, with seamless display in virtual reality (VR) wearables. The purpose of our study is to provide a system that makes use of cloud computing resources to offload the resource-intensive activities of 3D reconstruction and deep-learning-based scene interpretation. We establish an end-to-end pipeline from 2D to 3D reconstruction, which automatically builds accurate 3D models from collected photographs using sophisticated deep-learning techniques. These models are then converted to a VR-compatible format, allowing for immersive and interactive experiences on wearable devices. Our findings attest to the completion of 3D entities regenerated by the CAP–UDF model using ShapeNetCars and Deep Fashion 3D datasets with a discrepancy in L2 Chamfer distance of only 0.089 and 0.129, respectively. Furthermore, the demonstration of the end-to-end process from 2D capture to 3D visualization on VR occurs continuously.

Where Are We Now?—Exploring the Metaverse Representations to Find Digital Twins

“…Published By Year [46] MDPI 2023 [86] MDPI 2023 [24] IEEE 2023 [87] IAARC 2021 [56] ARXIV 2021 [31] MDPI 2023 [88] CellPress 2022 [47] Elsevier 2022 [27] Taylor and Francis 2019 [26] MDPI 2020 [89] IEEE 2022 [42] ASME 2023 [17] Elsevier 2022 [20] IEEE 2023 [90] IEEE 2022 [91] ACM 2023 [92] Elsevier 2023 [93] MDPI 2023 [55] JMIRx Med 2022 [94] MDPI 2023 [95] IEEE 2023 [96] IEEE 2023 [97] IEEE 2022 [98] Springer 2023…”