Wavelet Scattering Transform Depth Benefit, An Application for Speaker Identification

Moufidi, Abderrazzaq; Rousseau, David; Rasti, Pejman

doi:10.1007/978-3-031-20650-4_8

Cited by 1 publication

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the field of biometric identification, comprehensive studies have been conducted on unimodal systems [6,11,12]. This includes speaker identification through audio input [16,[19][20][21][22][23] as well as lip and facial identification via visual and behavioral factors [5][6][7]12,13]. However, each of these modes has potential drawbacks and limitations based on the conditions under which recordings are made.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Attention-Based Fusion of Ultrashort Voice Utterances and Depth Videos for Multimodal Person Identification

Moufidi

Rousseau

Rasti

2023

Sensors

Self Cite

View full text Add to dashboard Cite

Multimodal deep learning, in the context of biometrics, encounters significant challenges due to the dependence on long speech utterances and RGB images, which are often impractical in certain situations. This paper presents a novel solution addressing these issues by leveraging ultrashort voice utterances and depth videos of the lip for person identification. The proposed method utilizes an amalgamation of residual neural networks to encode depth videos and a Time Delay Neural Network architecture to encode voice signals. In an effort to fuse information from these different modalities, we integrate self-attention and engineer a noise-resistant model that effectively manages diverse types of noise. Through rigorous testing on a benchmark dataset, our approach exhibits superior performance over existing methods, resulting in an average improvement of 10%. This method is notably efficient for scenarios where extended utterances and RGB images are unfeasible or unattainable. Furthermore, its potential extends to various multimodal applications beyond just person identification.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Voice-based identification is another important aspect [11]. While it has been widely studied [11], acquiring long and clear voice samples can be challenging [14][15][16], espe-cially in real-world scenarios such as crowded places [17]. This necessitates the use of ultrashort voice utterances (less than 1 s) for identification.…”

Section: Introductionmentioning

confidence: 99%