Unified framework for face sketch synthesis

Wang, Nannan; Zhang, Shengchuan; Gao, Xinbo; Li, Jie; Song, Bin; Li, Zan

doi:10.1016/j.sigpro.2016.06.014

Cited by 19 publications

(6 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such technique can be divided into four categories, namely (1) entire face synthesis, (2) facial attributes manipulation, (3) face swap, (4) facial expression manipulation. The entire face synthesis technique [36] mainly focuses on creating a nonexistent face. With the successive appearance of deep learning methods [14], the entire face synthesis technique has the capacity to generate high-quality facial images which can hardly be distinguished through human eyes.…”

Section: Ai-based Facial Manipulation Algorithmsmentioning

confidence: 99%

Exposing DeepFake Videos Using Attention Based Convolutional LSTM Network

Xia

et al. 2021

Neural Process Lett

View full text Add to dashboard Cite

The detection of face tampering in videos created by artificial intelligence techniques (commonly known as the Deep Fakes) has become an important and challenging task in network security defense. In this paper, we propose a novel attention-based deep fake video detection method, which captures the sharp changes in terms of the facial features caused by the composite video. We utilize the convolutional long short-term memory to extract both spatial and temporal information of DeeFake videos. Meanwhile, we apply the attention mechanism to emphasize the specific facial area of each video frame. Finally, we design a decoder to further fusion multiple frames information for more accurate detection results. Experimental results and comparisons with state-of-the-art methods demonstrate that our framework achieves superior performance.

show abstract

Section: Ai-based Facial Manipulation Algorithmsmentioning

confidence: 99%

Exposing DeepFake Videos Using Attention Based Convolutional LSTM Network

Xia

et al. 2021

Neural Process Lett

View full text Add to dashboard Cite

show abstract

“…Some of the best-performing and most popular methods include Eigen-transformation (ET) that synthesises whole faces using a linear combination of photos (or sketches) under the assumption that face photos and the corresponding sketches are reasonably similar in appearance, the Eigen-patches (EP) extension in [11] to perform synthesis at a local level, the use of the Locally Linear Embedding (LLE) manifold learning technique in [19] to create a patch using a linear combination of neighbouring patches, the Multiscale Markov Random Fields (MRF) approach [20] which models the relationships among patches, its extension in [21] to cater specifically for lighting and pose variations, the Markov Weighted Fields (MWF) model in [22] that uses a weighted MRF to model the relation between photo and sketch patches, and the recent Bayesian framework proposed in [23] that considers relationships among neighbouring patches for neighbour selection and weight computation models. A more thorough review of FH algorithms may be found in [10], [23], and [24].…”

Section: A Face Photo-sketch Synthesis and Recognition Algorithmsmentioning

confidence: 99%

Matching Software-Generated Sketches to Face Photographs With a Very Deep CNN, Morphed Faces, and Transfer Learning

Galea

Farrugia

2018

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

Sketches obtained from eyewitness descriptions of criminals have proven to be useful in apprehending criminals, particularly when there is a lack of evidence. Automated methods to identify subjects depicted in sketches have been proposed in the literature, but their performance is still unsatisfactory when using software-generated sketches and when tested using extensive galleries with a large amount of subjects. Despite the success of deep learning in several applications including face recognition, little work has been done in applying it for face photograph-sketch recognition. This is mainly a consequence of the need to ensure robust training of deep networks by using a large number of images, yet limited quantities are publicly available. Moreover, most algorithms have not been designed to operate on software-generated face composite sketches which are used by numerous law enforcement agencies worldwide. This paper aims to tackle these issues with the following contributions: 1) a very deep convolutional neural network is utilised to determine the identity of a subject in a composite sketch by comparing it to face photographs and is trained by applying transfer learning to a state-of-the-art model pretrained for face photograph recognition; 2) a 3-D morphable model is used to synthesise both photographs and sketches to augment the available training data, an approach that is shown to significantly aid performance; and 3) the UoM-SGFS database is extended to contain twice the number of subjects, now having 1200 sketches of 600 subjects. An extensive evaluation of popular and stateof-the-art algorithms is also performed due to the lack of such information in the literature, where it is demonstrated that the proposed approach comprehensively outperforms state-of-the-art methods on all publicly available composite sketch datasets.

show abstract

“…using a linear combination of images, the Eigen-patches (EP) extension [15] performing synthesis at a local level, and the Bayesian framework in [16] that considers relationships among neighbouring patches for model construction. A more thorough review of FH algorithms may be found in [4], [13], [15]- [17]. State-of-the-art inter-modality methods that learn or extract modality-invariant features include the D-RS approach [2], [18] that compares SIFT and MLBP descriptors extracted from images that are convolved with three filters, the CBR method [19], which compares MLBP features extracted from individual facial components, the FaceSketchID system in [12] which fuses D-RS with CBR, and the recent LGMS method [4] that compares MLBP features extracted from log-Gabor-filtered images using the spearman rank-order correlation coefficient.…”

Section: Related Workmentioning

confidence: 99%

Forensic Face Photo-Sketch Recognition Using a Deep Learning-Based Architecture

Galea

Farrugia

2017

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

Numerous methods that automatically identify subjects depicted in sketches as described by eyewitnesses have been implemented, but their performance often degrades when using real-world forensic sketches and extended galleries that mimic law enforcement mug-shot galleries. Moreover, little work has been done to apply deep learning for face photo-sketch recognition despite its success in numerous application domains including traditional face recognition. This is primarily due to the limited number of sketch images available, which are insufficient to robustly train large networks. This letter aims to tackle these issues with the following contributions: 1) a state-of-the-art model pre-trained for face photo recognition is tuned for face photo-sketch recognition by applying transfer learning, 2) a three-dimensional morphable model is used to synthesise new images and artificially expand the training data, allowing the network to prevent over-fitting and learn better features, 3) multiple synthetic sketches are also used in the testing stage to improve performance, and 4) fusion of the proposed method with a state-of-the-art algorithm is shown to further boost performance. An extensive evaluation of several popular and state-of-the-art algorithms is also performed using publicly available datasets, thereby serving as a benchmark for future algorithms. Compared to a leading method, the proposed framework is shown to reduce the error rate by 80.7% for viewed sketches and lowers the mean retrieval rank by 32.5% for real-world forensic sketches.

show abstract

Unified framework for face sketch synthesis

Cited by 19 publications

References 29 publications

Exposing DeepFake Videos Using Attention Based Convolutional LSTM Network

Exposing DeepFake Videos Using Attention Based Convolutional LSTM Network

Matching Software-Generated Sketches to Face Photographs With a Very Deep CNN, Morphed Faces, and Transfer Learning

Forensic Face Photo-Sketch Recognition Using a Deep Learning-Based Architecture

Contact Info

Product

Resources

About