3D Vision Using Multiple Structured Light-Based Kinect Depth Cameras

Abstract

Real-time 3D scanning of a scene or object using multiple depth cameras is often required in many applications but is still a challenging task for the computer vision community, especially when the object or scene is partially occluded and dynamic. If active depth sensors are used in this case, their resulting depth map quality gets degraded due to interference between active radiations from each depth sensor. Passive 3D sensors like stereo cameras can avoid the issue of interference as they do not emit any active radiation, but they face correspondence problems. Since releasing the commodity depth sensor Microsoft Kinect, researchers are getting more interested in active depth-sensing. However, Kinect sensors have some easily noticeable limitations concerning 3D reconstruction such as: they can provide depth maps for a limited range, their field of view is restricted and holes are observed in the depth map due to occlusion. The above-mentioned limitations can be overcome if multiple Kinect sensors are used simultaneously instead of a single Kinect sensor. Still, the challenge here is to avoid interference between these sensors. We present a comprehensive review of possible solutions to avoid interference between multiple Kinect sensors. Furthermore, we introduce the Kinect technology in detail along with applications where multiple Kinect sensors are used in the literature. We expect that this paper will be helpful to the researchers who want to use multiple Kinect sensors in sharing the workplace in their research.

Design and analysis of solid ankle foot orthosis by employing mechanical characterization and a low-cost scanning approach for additive manufacturing

“…The infrared projector emits a speckle pattern of infrared dots into the camera's field of view. As a result of Kinect's depth sensor technology, the Kinect sensor can create 3D maps of objects within its range by measuring changes to the reference speckle pattern (Kamble & Mahajan, 2022). A Kinect sensor can capture color and depth images simultaneously at a frame rate of 30 frames per second.…”

4D Feet: Registering Walking Foot Shapes Using Attention Enhanced Dynamic-Synchronized Graph Convolutional LSTM Network

“…where Γ is known as the growth rate of the estimation law. Using (20), the instruction of ϕ and φ, and φ → 0, we may conclude that Π → A M , and…”

“…We set the initial learning rate to 0.0005 which is reduced in every 15 epochs by multiplying 0.1 to the learning rate. In addition, we discretize the parameter estimation formula in (20) by considering φ = φ(k+1)− φ(k) ∆k , k ∈ Z ≥0 ; then, knowing that ∆k = 1 as k is a sequentially increasing index (the index of intervals in the registration process), the estimation rule (20) turns into…”

“…In addition, "dropped frames" are frequently observed in the captured data, mainly due to that the huge amount of to-be-transferred data leads to a nonlinear accumulative delay regarding each camera [30]. Relevant literature on 3D reconstruction from asynchronise multi-views studied in [20], [29], [31] show the importance of a general timesynchroniser method for collecting any 4D data-set through multiple cameras. In a practical case of using 6 Azure Kinect DK cameras for 15 fps 4D scanning, as in FIGURE 1, even when all cameras are hardware synchronized, we found that there are on average 2 ms delays for each frame acquired by those cameras in a 3-seconds scanning session.…”

Indoor Environment Reconstruction for Unmanned System Using Multiple Low-Cost Sensors

“…The general process can be broken down into the following steps: feature points extraction, image matching, dense point cloud achievement, surface reconstruction and textured mapping [11]. Nevertheless, vision-based systems are limited to reconstructing an environment with low illumination and sparse or repetitive features, due to the problem of calculating depth by finding correspondence between two images [12].…”