Attention-Driven Cropping for Very High Resolution Facial Landmark Detection

Abstract

Facial landmark detection is a fundamental task for many consumer and high-end applications and is almost entirely solved by machine learning methods today. Existing datasets used to train such algorithms are primarily made up of only low resolution images, and current algorithms are limited to inputs of comparable quality and resolution as the training dataset. On the other hand, high resolution imagery is becoming increasingly more common as consumer cameras improve in quality every year. Therefore, there is need for algorithms that can leverage the rich information available in high resolution imagery. Naïvely attempting to reuse existing network architectures on high resolution imagery is prohibitive due to memory bottlenecks on GPUs. The only current solution is to downsample the images, sacrificing resolution and quality. Building on top of recent progress in attention-based networks, we present a novel, fully convolutional regional architecture that is specially designed for predicting landmarks on very high resolution facial images without downsampling. We demonstrate the flexibility of our architecture by training the proposed model with images of resolutions ranging from 256 x 256 to 4K. In addition to being the first method for facial landmark detection on high resolution images, our approach achieves superior performance over traditional (holistic) state-of-the-art architectures across ALL resolutions, leading to a general-purpose, extremely flexible, high quality landmark detector.

Robust facial landmark detection by cross-order cross-semantic deep network

“…Implementation Details In our experiments, we first use the four stacked hourglass networks as our backbone [37]. All the training and testing images are In the next section, we firstly compare our algorithm with the state-of-theart methods, such as 3DDFA [44], RAR [29], Wing [36], LAB [8], DU-Net [48], Liu et al [11], ODN [9], HRNet [49], Chandran et al [13] and LUVLi [12].…”

“…For instance, the coordinate regression facial landmark detection methods [6,8,9] learn features from the whole face images and then regress to the landmark coordinates, which drives the models to learn the whole facial features in a common/normal way that cannot accurately model the differences of local details and the relationships among local details. Also, the heatmap regression facial landmark detection methods [10,11,12,13] generate a landmark heatmap for each landmark and then predict landmarks by traversing the corresponding landmark heatmaps.…”

D’OraCa: Deep Learning-Based Classification of Oral Lesions with Mouth Landmark Guidance for Early Detection of Oral Cancer

“…With the fast development of deep learning techniques in computer vision, deep learning based methods [9,11,21,27,46,47] have significantly boosted and outperformed both the template fitting method and cascaded regression-based method, creating a new state-of-the-art in facial landmark detection task. Most of them leverage deep convolutional neural networks (CNN) to learn facial features and predict the facial landmark in an end-to-end fashion.…”

A Keypoint Transformer to Discover Spine Structure for Cobb Angle Estimation

“…Recently, landmark detection on faces and bodys has attracted great attention from researchers, and methods in this area can be roughly divided into five branches: detection from global input to local regions [7,8], direct localization [9], auxiliary inputs [10,11,12], and feature refinement [13,14,15,16].…”

“…Specifically, landmarks are detected with downsampled images using an hourglass network and refined with cropped local regions from input images using similar models [7]; low-resolution response maps are generated using a fully convolutional network, and high-resolution ones over local regions are produced to output more accurate landmarks [8]. CenterNet directly outputs landmarks from one center keypoint heatmap and two corner ones with a convolutional backbone [9].…”