3D Face Point Cloud Reconstruction and Recognition Using Depth Sensor

Abstract

Facial recognition has attracted more and more attention since the rapid growth of artificial intelligence (AI) techniques in recent years. However, most of the related works about facial reconstruction and recognition are mainly based on big data collection and image deep learning related algorithms. The data driven based AI approaches inevitably increase the computational complexity of CPU and usually highly count on GPU capacity. One of the typical issues of RGB-based facial recognition is its applicability in low light or dark environments. To solve this problem, this paper presents an effective procedure for facial reconstruction as well as facial recognition via using a depth sensor. For each testing candidate, the depth camera acquires a multi-view of its 3D point clouds. The point cloud sets are stitched for 3D model reconstruction by using the iterative closest point (ICP). Then, a segmentation procedure is designed to separate the model set into a body part and head part. Based on the segmented 3D face point clouds, certain facial features are then extracted for recognition scoring. Taking a single shot from the depth sensor, the point cloud data is going to register with other 3D face models to determine which is the best candidate the data belongs to. By using the proposed feature-based 3D facial similarity score algorithm, which composes of normal, curvature, and registration similarities between different point clouds, the person can be labeled correctly even in a dark environment. The proposed method is suitable for smart devices such as smart phones and smart pads with tiny depth camera equipped. Experiments with real-world data show that the proposed method is able to reconstruct denser models and achieve point cloud-based 3D face recognition.

Deep Learning Models for Automatic Upper Airway Segmentation and Minimum Cross-Sectional Area Localisation in Two-Dimensional Images

“…It is also extremely time-consuming to manually determine CSAmin, a crucial parameter in airway assessment, in three dimensions; the judgement cannot be based solely on 2D images because data from the third dimension are lacking. However, CNNs can predict 3D information from 2D data after realising a large amount of 3D feature data [ 24 , 25 ]. We utilised this strength of CNN deep learning algorithms and successfully developed a fully automatic AI-driven system for upper airway segmentation and CSAmin localisation using 2D radiographic airway images.…”

“…But CSAmin localisation requires clinicians to reconstruct the upper airway structure in three dimensions and compare the area values across different planes. In recent AI studies, CNNs were used to successfully infer or reconstruct 3D structures based on 2D data after realising large amounts of 3D features and data [ 24 , 25 ]; this would have been impossible to achieve using manual processing (owing to a lack of data from the third dimension).…”

Deep Learning-Based Standardized Evaluation and Human Pose Estimation: A Novel Approach to Motion Perception

“…Usually, there are two ways to obtain skeleton points: one is to obtain point cloud data through special devices such as depth sensors [22], and then perform pose recognition through the point cloud, to obtain the skeleton and key points, which can ensure high recognition accuracy, but relying on special equipment with a huge amount of computation, and has a depth-of-field limitation, being unfavorable to the environment of multiple dancing [23]; the other one is realized by ordinary camera such as OpenPose pose recognition algorithm, which can obtain high accuracy under various working conditions with no need of special equipment, and also gradually becomes the mainstream scheme of pose recognition.…”

Effective Key Region-Guided Face Detail Optimization Algorithm for 3D Face Reconstruction

“…where dx and dy are the lengths that pixels occupy in the X and Y directions, respectively, and ðu 0 , v 0 Þ is the origin of the image. The conversion of the image coordinate system to the camera coordinate system involves a perspective projection (Figure 4(b)), and it can be expressed as Equation (2).…”

“…For instance, virtual teleconference software uses reconstructed heads to compress the transmission bandwidth [1]. Security and electronic payment systems utilize facial depth characteristics to validate user identities [2]. In the medical field, surgical planning and analysis, as well as communication between doctors and patients, have become more efficient with the interactive assistance of virtual faces [3,4].…”