Sensing-by-Overlaying: A Practical Implementation of a Multiplayer Mixed-Reality Gaming System by Integrating a Dense Point Cloud and a Real-Time Camera

Kurabayashi, Shuichi; Ishiyama, Hidetoshi; Kanai, Masaru

doi:10.1109/ism.2016.0136

Cited by 4 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, point cloud registration is a fundamental process for all industries that require complete 3D point cloud data. A complete point cloud provides a more accurate and detailed digital representation of the real world and is commonly used in gaming ( Kurabayashi et al., 2016 ), as well as inspection and visualization in construction ( Wang and Kim, 2019 ). Notably, point clouds provide the ability to manipulate the model in full 3D post-acquisition.…”

Section: Introductionmentioning

confidence: 99%

Point cloud completion in challenging indoor scenarios with human motion

Zhang

Czarnuch

2023

Front. Robot. AI

View full text Add to dashboard Cite

Combining and completing point cloud data from two or more sensors with arbitrarily relative perspectives in a dynamic, cluttered, and complex environment is challenging, especially when the two sensors have significant perspective differences while the large overlap ratio and feature-rich scene cannot be guaranteed. We create a novel approach targeting this challenging scenario by registering two camera captures in a time series with unknown perspectives and human movements to easily use our system in a real-life scene. In our approach, we first reduce the six unknowns of 3D point cloud completion to three by aligning the ground planes found by our previous perspective-independent 3D ground plane estimation algorithm. Subsequently, we use a histogram-based approach to identify and extract all the humans from each frame generating a three-dimensional (3D) human walking sequence in a time series. To enhance accuracy and performance, we convert 3D human walking sequences to lines by calculating the center of mass (CoM) point of each human body and connecting them. Finally, we match the walking paths in different data trials by minimizing the Fréchet distance between two walking paths and using 2D iterative closest point (ICP) to find the remaining three unknowns in the overall transformation matrix for the final alignment. Using this approach, we can successfully register the corresponding walking path of the human between the two cameras’ captures and estimate the transformation matrix between the two sensors.

show abstract

Section: Introductionmentioning

confidence: 99%