End-to-End Spectro-Temporal Graph Attention Networks for Speaker Verification Anti-Spoofing and Speech Deepfake Detection

Tak, Hemlata; Jung, Jee-weon; Patino, José; Kamble, Madhu R.; Todisco, Massimiliano; Evans, Nicholas

doi:10.48550/arxiv.2107.12710

Cited by 12 publications

(17 citation statements)

References 54 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the ASV models, we use Resnet34 [38], ECAPA-TDNN [35] and MFA-Conformer [39]. For the countermeasure models, we use AASIST [17], AASIST-L, and RawGAT-ST [25], where AASIST-L is a light version of AASIST. The fusion model in Figure 1 is trained by Adam optimizer with an initial learning rate of 0.0001.…”

Section: Methodsmentioning

confidence: 99%

“…The current solutions leverage end-toend deep neural networks (DNNs) [12,13], trying to distinguish artifacts and unnatural cues of spoofing speech from bona fide speech. And thanks to a series of challenges and datasets [1][2][3][4], many novel techniques were introduced to achieve promising CM performances [12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Spoofing-Aware Speaker Verification by Multi-Level Fusion

Wu¹,

Meng²,

Kang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recently, many novel techniques have been introduced to deal with spoofing attacks, and achieve promising countermeasure (CM) performances. However, these works only take the standalone CM models into account. Nowadays, a spoofing aware speaker verification (SASV) challenge which aims to facilitate the research of integrated CM and ASV models, arguing that jointly optimizing CM and ASV models will lead to better performance, is taking place. In this paper, we propose a novel multi-model and multi-level fusion strategy to tackle the SASV task. Compared with purely scoring fusion and embedding fusion methods, this framework first utilizes embeddings from CM models, propagating CM embeddings into a CM block to obtain a CM score. In the second-level fusion, the CM score and ASV scores directly from ASV systems will be concatenated into a prediction block for the final decision. As a result, the best single fusion system has achieved the SASV-EER of 0.97% on the evaluation set. Then by ensembling the top-5 fusion systems, the final SASV-EER reached 0.89%.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Spoofing-Aware Speaker Verification by Multi-Level Fusion

Wu¹,

Meng²,

Kang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We found that a number of researches have investigated various acoustic features to demonstrate their robustness to PA and LA with the state-of-the-art backend neural networks including: ResNet [13][14][15][16][17], Res2Net [18,19], and graph attention network [20,21]. The representative acoustic features for SSD are: log-power (or magnitude) discrete Fourier transform (DFT), constant-Q transform (CQT), and linear frequency cepstral coefficients (LFCC), which are the magnitude features in the frequency domain [16,18,[22][23][24].…”

Section: Introductionmentioning

confidence: 99%

Phase-Aware Spoof Speech Detection Based on Res2Net with Phase Network

Kim¹,

Ban²

2022

Preprint

View full text Add to dashboard Cite

The spoof speech detection (SSD) is the essential countermeasure for automatic speaker verification systems. Although SSD with magnitude features in the frequency domain has shown promising results, the phase information also can be important to capture the artefacts of certain types of spoofing attacks. Thus, both magnitude and phase features must be considered to ensure the generalization ability to diverse types of spoofing attacks. In this paper, we investigate the failure reason of feature-level fusion of the previous works through the entropy analysis from which we found that the randomness difference between magnitude and phase features is large, which can interrupt the feature-level fusion via backend neural network; thus, we propose a phase network to reduce that difference. Our SSD system: phase network equipped Res2Net achieved significant performance improvement, specifically in the spoofing attack for which the phase information is considered to be important. Also, we demonstrate our SSD system in both knownand unknown-kind SSD scenarios for practical applications.

show abstract

“…These end-toend models can also be viewed as a category of methods that replace the aforementioned feature extraction with neural nets with trainable parameters. Several studies utilized sinc convolution-based network [20] as a front-end of anti-spoofing model [17,21].…”

Section: Introductionmentioning

confidence: 99%

Representation Selective Self-distillation and wav2vec 2.0 Feature Exploration for Spoof-aware Speaker Verification

Lee,

Kim,

Koo

et al. 2022

Preprint

View full text Add to dashboard Cite

Text-to-speech and voice conversion studies are constantly improving to the extent where they can produce synthetic speech almost indistinguishable from bona fide human speech. In this regrad, the importance of countermeasures (CM) against synthetic voice attacks of the automatic speaker verification (ASV) systems emerges. Nonetheless, most end-to-end spoofing detection networks are black box systems, and the answer to what is an effective representation for finding artifacts still remains veiled. In this paper, we examine which feature space can effectively represent synthetic artifacts using wav2vec 2.0, and study which architecture can effectively utilize the space. Our study allows us to analyze which attribute of speech signals is advantageous for the CM systems. The proposed CM system achieved 0.31% equal error rate (EER) on ASVspoof 2019 LA evaluation set for the spoof detection task. We further propose a simple yet effective spoofing aware speaker verification (SASV) methodology, which takes advantage of the disentangled representations from our countermeasure system. Evaluation performed with the SASV Challenge 2022 database show 1.08% of SASV EER. Quantitative analysis shows that using the explored feature space of wav2vec 2.0 advantages both spoofing CM and SASV.

show abstract

End-to-End Spectro-Temporal Graph Attention Networks for Speaker Verification Anti-Spoofing and Speech Deepfake Detection

Cited by 12 publications

References 54 publications

Spoofing-Aware Speaker Verification by Multi-Level Fusion

Spoofing-Aware Speaker Verification by Multi-Level Fusion

Phase-Aware Spoof Speech Detection Based on Res2Net with Phase Network

Representation Selective Self-distillation and wav2vec 2.0 Feature Exploration for Spoof-aware Speaker Verification

Contact Info

Product

Resources

About