Domain Alignment Embedding Network for Sketch Face Recognition

Guo, Yanan; Cao, Lin; Chen, Changwu; Du, Kangning; Fu, Chong

doi:10.1109/access.2020.3047108

Cited by 5 publications

(37 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the first two setup, performance of the CTMAN with the CTMAN*, CTMAN-ResNet18, PCA (Turk, 1991), ET(+PCA) (Tang and Wang, 2004), EP(+PCA) (Galea and Farrugia, 2015), LLE(+PCA) (Chang et al, 2004), CBR (Hu et al, 2013), D-RS (Klare and Jain, 2015), CBR+D-RS (Klare and Jain, 2015), LGMS (Galea and Farrugia, 2016), HAOG (Galoogahi and Sim, 2012), VGG-Face (Parkhi et al, 2015), DEEPS (Galea and Farrugia, 2018), Xu's (Xu et al, 2021), DLFace (Peng et al, 2019), SSR (Peng et al, 2021), andDAEN (Guo et al, 2021) methods are reported in Tables 5, 6. The performance of these compared approaches is directly from Galea and Farrugia (2018), Xu et al (2021), Peng et al (2019), Peng et al (2021), andGuo et al (2021). The extended gallery set in Galea and Farrugia (2018) consists of part images of the FEI, MEDS-II, Multi-PIE (Gross et al, 2010), and FRGC v2.0 4 datasets, these images are frontal and have high quality.…”

Section: Comparison To the State-of-the-art Methodsmentioning

confidence: 99%

“…For the fourth setup, the performance of the CTMAN with the SSD (Mittal et al, 2014), Attribute (Mittal et al, 2017), Transfer Learning (Mittal et al, 2015), and DAEN (Guo et al, 2021 methods are reported in Table 8. The performance of these compared approaches are directly from Mittal et al (2015), Mittal et al (2017), andGuo et al (2021). The SSD and Attribute are traditional methods, whereas Transfer Learning and DAEN are deep learning methods.…”

Section: Comparison To the State-of-the-art Methodsmentioning

confidence: 99%

“…By using a 3-D morphable model to synthesize both photos and sketches to augment the training data, Galea and Farrugia (2018) utilized a fine-tuned VGG-Face network and a triplet loss to determine the identity in a query sketch by comparing it to a gallery set. Guo et al (2021) designed a training episode strategy to alleviate the small sample problem and proposed a domain alignment embedding loss to guide the network to learn discriminative features. Recently, few-shot learning has become appealing choice to deal with a small sample problem.…”

Section: Related Workmentioning

confidence: 99%

“…Due to the privacy protection problems and the time consuming efforts of sketch drawing, amount of the paired sketch-photo data is limited. Inspired by the few shot learning methods (Vinyals et al, 2016;Snell et al, 2017;Jiang et al, 2018;Guo et al, 2021), a meta learning training episode strategy is introduced to incorporate modality information by sampling image pairs and classes from the training set.…”

Section: Meta Learning Episode Training Strategymentioning

confidence: 99%

“…Note that CTMAN is different from other sketch face recognition schemes, such as Domain Alignment Embedding Network (DAEN) (Guo et al, 2021). The main differences between the CTMAN and the DAEN are as follows: (1) CTMAN uses a two-stream network to extract discriminative modality alignment feature, the two-stream network consists of a ResNet50 backbone, the non-local blocks and the generalized mean (GeM) pooling layers.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Cross Task Modality Alignment Network for Sketch Face Recognition

Guo

Cao

2022

Front. Neurorobot.

Self Cite

View full text Add to dashboard Cite

The task of sketch face recognition refers to matching cross-modality facial images from sketch to photo, which is widely applied in the criminal investigation area. Existing works aim to bridge the cross-modality gap by inter-modality feature alignment approaches, however, the small sample problem has received much less attention, resulting in limited performance. In this paper, an effective Cross Task Modality Alignment Network (CTMAN) is proposed for sketch face recognition. To address the small sample problem, a meta learning training episode strategy is first introduced to mimic few-shot tasks. Based on the episode strategy, a two-stream network termed modality alignment embedding learning is used to capture more modality-specific and modality-sharable features, meanwhile, two cross task memory mechanisms are proposed to collect sufficient negative features to further improve the feature learning. Finally, a cross task modality alignment loss is proposed to capture modality-related information of cross task features for more effective training. Extensive experiments are conducted to validate the superiority of the CTMAN, which significantly outperforms state-of-the-art methods on the UoM-SGFSv2 set A, set B, CUFSF, and PRIP-VSGC dataset.

show abstract

Section: Comparison To the State-of-the-art Methodsmentioning

confidence: 99%

Section: Comparison To the State-of-the-art Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Meta Learning Episode Training Strategymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Cross Task Modality Alignment Network for Sketch Face Recognition

Guo

Cao

2022

Front. Neurorobot.

Self Cite

View full text Add to dashboard Cite

show abstract

Residual serialized cross grouping transformer for small scale sketch face recognition

Du,

Wang,

Yin

et al. 2024

Complex Intell. Syst.

View full text Add to dashboard Cite

Sketch face recognition has recently gained significant attention in the field of computer vision due to its ability to quickly identify matched pairs of optical and sketch images. This technology has the potential to greatly improve the efficiency of law enforcement agencies in criminal investigations. However, there are still challenges that need to be addressed in sketch face recognition algorithms, such as modal differences and limited sample sizes. To overcome these issues, this study proposes a Residual Serialized Cross Grouping Transformer (RSCGT), which contains a residual serialized module to reduce the computation complexity, a two-layer Cross Grouping Transformer module that is capable of extracting modality-invariant context features, a domain adaptive module to mitigate the impact of modal differences. Additionally, we introduce a meta-learning training strategy to augment the generalization ability of this model. Experimental results demonstrate that the RSCGT achieves high accuracy in sketch face recognition tasks, even with small-scale datasets.

show abstract

Sketch face recognition based on light semantic Transformer network

Cao

Yin

Guo

et al. 2023

IET Computer Vision

Self Cite

View full text Add to dashboard Cite

Sketch face recognition has a wide range of applications in criminal investigation, but it remains a challenging task due to the small‐scale sample and the semantic deficiencies caused by cross‐modality differences. The authors propose a light semantic Transformer network to extract and model the semantic information of cross‐modality images. First, the authors employ a meta‐learning training strategy to obtain task‐related training samples to solve the small sample problem. Then to solve the contradiction between the high complexity of the Transformer and the small sample problem of sketch face recognition, the authors build the light semantic transformer network by proposing a hierarchical group linear transformation and introducing parameter sharing, which can extract highly discriminative semantic features on small–scale datasets. Finally, the authors propose a domain‐adaptive focal loss to reduce the cross‐modality differences between sketches and photos and improve the training effect of the light semantic Transformer network. Extensive experiments have shown that the features extracted by the proposed method have significant discriminative effects. The authors’ method improves the recognition rate by 7.6% on the UoM‐SGFSv2 dataset, and the recognition rate reaches 92.59% on the CUFSF dataset.

show abstract

Domain Alignment Embedding Network for Sketch Face Recognition

Cited by 5 publications

References 49 publications

Cross Task Modality Alignment Network for Sketch Face Recognition

Cross Task Modality Alignment Network for Sketch Face Recognition

Residual serialized cross grouping transformer for small scale sketch face recognition

Sketch face recognition based on light semantic Transformer network

Contact Info

Product

Resources

About