Facial Component-Landmark Detection With Weakly-Supervised LR-CNN

Zhang, Ruiheng; Mu, Chengpo; Xu, Min; Xu, Lixin; Xu, Xiaofeng

doi:10.1109/access.2018.2890573

Cited by 10 publications

(8 citation statements)

References 49 publications

(70 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the deep learning-based methods have shown promising results in both landmark detection [29], [30] and area segmentation tasks [21], [42], one of the possible reasons for limited studies on auricle-related topics might be the missing of ear image datasets with annotated landmarks. One closely related research topic is facial landmark detection, and many studies localize landmark points by utilizing deep learning technology, including various of CNN-based networks [22], [24], [27], multitask learning [23], [25], and transform learning [28] etc. For example, Sun et al [22] proposed a three-level cascaded convolutional network to estimate the positions of facial keypoints.…”

Section: Related Workmentioning

confidence: 99%

Deep Learning-Based Auricular Point Localization for Auriculotherapy

Sun

Dong

Li³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Auriculotherapy is one of the main forms of treatment in Traditional Chinese Medicine, whose potential as an alternative medicine for both health evaluation and disease treatment has been reported in many cases. However, its efficacy highly relies on the accurate localization of auricular points, which are not easy to be remembered due to their complexity. To explore an efficient way of locating auricular points, this study proposed a deep learning-based method of automatically locating auricular points from auricular images. A self-collected dataset named EID was created for TCM auriculotherapy research, with 91 auriculotherapy-related landmark points manually annotated according to the Chinese national standardization. A deep neural network structure was trained for landmark detection, and a direction normalization module was proposed to compensate for the detection error caused by the difference between the left and right ears. The trained model was validated on dataset EID. An average NME of 0.0514±0.0023 was achieved, which outperformed similar works. In addition, a certain auricular area corresponding to the digestive system was segmented based on the localized landmarks, and the results were tested in real-time video streaming. The proposed work for both auricular landmark and area identification can be widely used in auriculotherapy education and applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Learning-Based Auricular Point Localization for Auriculotherapy

Sun

Dong

Li³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Several graphics rendering engines are well-suited to infrared simulation [18], [21], [28], [29]. For instance, Lahoud et al [18] use Unity3D and Oculus Rift to build an IR augmented reality system, which can simulate a thermal camera.…”

Section: Related Work a Ir Simulator And Synthetic-based Systemmentioning

confidence: 99%

“…In addition, through calculating the radiation model, object-oriented graphics rendering engine (OGRE) is also utilized to simulate a real three-dimensional infrared complex scene, which is developed by [29]. On the basis of Mu's work, Gao and Zhang [21]- [23] design an infrared scene simulator to generate thousands of simulated IR images, which can be used to train classifiers and detectors [28]. Meanwhile, for the scene simulation, Guo et al [24] produce a semi-automatic system to simulate large-scale IR urban scenes in the form of levels of detail.…”

Section: Related Work a Ir Simulator And Synthetic-based Systemmentioning

confidence: 99%

“…Thus, the idea of utilizing synthetic instead of real IR images has become appealing, because the synthetic IR images can be mass generated and easily annotated. Several graphics rendering engines are well-suited to infrared simulation [18], [21], [28], [29]. Many efforts have explored using these generated synthetic data for various prediction tasks, including radar data classification [17], IR target detection [21], and semantic segmentation [20].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Synthetic IR Image Refinement Using Adversarial Learning With Bidirectional Mappings

Zhang

et al. 2019

IEEE Access

Self Cite

View full text Add to dashboard Cite

Collecting a large dataset of real infrared (IR) images is expensive, time-consuming, and even unavailable in some specific scenarios. With recent progress in machine learning, it has become more feasible to replace real IR images with qualified synthetic IR images in learning-based IR systems. However, this alternative may fail to achieve the desired performance, due to the gap between real and synthetic IR images. Inspired by adversarial learning for image-to-image translation, we propose the Synthetic IR Refinement Generative Adversarial Network (SIR-GAN) to narrow this gap. By learning the bidirectional mappings between two unpaired domains, the realism of the simulated IR images generated from the IR Simulator are significantly improved, where the source domain contains a large number of simulated IR images, where the target domain contains a limited quantity of real IR images. Specifically, driven by the idea of transferring infrared characteristic and protect target semantic information simultaneously, we propose a SIR refinement loss to consider an infrared loss and a structure loss further to the adversarial loss and the consistency loss. To further reduce the gap, stabilize training, and avoid artefacts, we modify the proposed algorithm by developing a training strategy, adding the U-net in the generators, using the dilated convolution in the discriminators and invoking the N-Adam acts as the optimizer. Qualitative, quantitative, and ablation study experiments demonstrate the superiority of the proposed approach compared with the state-of-the-art techniques in terms of realism and fidelity. In addition, our refined IR images are evaluated in the context of a feasibility study, where the accuracy of the trained classifier is significantly improved by adding our refined data into a small real-data training set.

show abstract

“…More recently, DNNs have become the trend in face alignment. [33][34][35][36][37][38] For example, Feng et al proposed a Wing loss function for convolutional neural network (CNN)-based face alignment, which improves the performance of regression-based face alignment with CNNs significantly. 39 In this article, we use a modified regression visual geometry group (VGG) architecture to obtain fine-grained geometric facial features in the form of a set of 2D facial landmarks.…”

Section: Introductionmentioning

confidence: 99%

Deep aligned feature extraction for collaborative-representation-based face classification with group dictionary selection

Mao

Delei

Chen

et al. 2020

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Face recognition plays an important role in many robotic and human–computer interaction systems. To this end, in recent years, sparse-representation-based classification and its variants have drawn extensive attention in compress sensing and pattern recognition. For image classification, one key to the success of a sparse-representation-based approach is to extract consistent image feature representations for the images of the same subject captured under a wide spectrum of appearance variations, for example, in pose, expression and illumination. These variations can be categorized into two main types: geometric and textural variations. To eliminate the difficulties posed by different appearance variations, the article presents a new collaborative-representation-based face classification approach using deep aligned neural network features. To be more specific, we first apply a facial landmark detection network to an input face image to obtain its fine-grained geometric information in the form of a set of 2D facial landmarks. These facial landmarks are then used to perform 2D geometric alignment across different face images. Second, we apply a deep neural network for facial image feature extraction due to the robustness of deep image features to a variety of appearance variations. We use the term deep aligned features for this two-step feature extraction approach. Last, a new collaborative-representation-based classification method is used to perform face classification. Specifically, we propose a group dictionary selection method for representation-based face classification to further boost the performance and reduce the uncertainty in decision-making. Experimental results obtained on several facial landmark detection and face classification data sets validate the effectiveness of the proposed method.

show abstract

Facial Component-Landmark Detection With Weakly-Supervised LR-CNN

Cited by 10 publications

References 49 publications

Deep Learning-Based Auricular Point Localization for Auriculotherapy

Deep Learning-Based Auricular Point Localization for Auriculotherapy

Synthetic IR Image Refinement Using Adversarial Learning With Bidirectional Mappings

Deep aligned feature extraction for collaborative-representation-based face classification with group dictionary selection

Contact Info

Product

Resources

About