Bi-Modal Person Recognition on a Mobile Phone: Using Mobile Phone Data

McCool, Chris; Marcel, Sébastien; Hadid, Abdenour; Pietikäinen, Matti; Matějka, Pavel; #x, Jan Cernock; Poh, Norman; Kittler, Josef; Larcher, Anthony; Lévy, Christophe; Matrouf, Driss; Bonastre, Jean-François; Tresadern, Philip A.; Cootes, Timothy F.

doi:10.1109/icmew.2012.116

Cited by 189 publications

(148 citation statements)

References 14 publications

Supporting

Mentioning

147

Contrasting

Unclassified

Order By: Relevance

“…On the other hand, this gives us the opportunity to test our system on a recently published challenging dataset that satisfies our experimental setting choices. The proposed framework is evaluated in a subject-independent fashion on native and non-native speech episodes of English speech from the MOBIO Database [24].…”

Section: Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Discrimination Between Native and Non-Native Speech Using Visual Features Only

Georgakis

Petridis

Pantić

2016

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Abstract-Accent is a soft biometric trait that can be inferred from pronunciation and articulation patterns characterising the speaking style of an individual. Past research has addressed the task of classifying accent, as belonging to a native language speaker or a foreign language speaker, by means of the audio modality only. However, features extracted from the visual stream of speech have been successfully used to extend or substitute audio-only approaches that target speech or language recognition. Motivated by these findings, we investigate to what extent temporal visual speech dynamics attributed to accent can be modelled and identified when the audio stream is missing or noisy, and the speech content is unknown. We present here a fully automated approach to discriminating native from non-native English speech, based exclusively on visual cues. A systematic evaluation of various appearance and shape features for the target problem is conducted, with the former consistently yielding superior performance. Subject-independent cross-validation experiments are conducted on mobile phone recordings of continuous speech and isolated word utterances spoken by 56 subjects from the challenging MOBIO Database. High performance is achieved on a text-dependent protocol, with the best score of 76.5% yielded by fusion of five HMMs trained on appearance features. Our framework is also efficient even when tested on examples of speech unseen in the training phase, although performing less accurately compared to the text-dependent case.

show abstract

Section: Datasetmentioning

confidence: 99%

“…Speaker-independent accent classification experiments are conducted first on continuous reading speech samples from the MOBIO Database [24], all captured by mobile phones. In that experiment, the experimental scenario is text-dependent, i.e., all speakers utter the same three-sentence paragraph.…”

Section: Introductionmentioning

confidence: 99%

Discrimination Between Native and Non-Native Speech Using Visual Features Only

Georgakis

Petridis

Pantić

2016

IEEE Trans. Cybern.

View full text Add to dashboard Cite

show abstract

“…We have used four publicly available datasets, namely, MOBIO [36], REPLAY-ATTACK [37], MSU-MFSD [38], and REPLAY-MOBILE [39].…”

Section: Datasetsmentioning

confidence: 99%

“…The MOBIO dataset [36] was collected for bi-modal (voice and face) biometric verification experiments using mobile devices. Therefore, it contains only licit protocols.…”

Section: Datasetsmentioning

confidence: 99%

Deeply vulnerable: a study of the robustness of face recognition to presentation attacks

2017

View full text Add to dashboard Cite

The vulnerability of deep-learning based face-recognition (FR) methods, to presentation attacks (PA) is studied in this paper. Recently proposed FR methods based on deep neural networks (DNN) have been shown to outperform most other methods by a significant margin. In a trustworthy face-verification system, however, maximizing recognition-performance alone is not sufficient -the system should also be capable of resisting various kinds of attacks, including presentation-attacks (PA). Previous experience has shown that the PA-vulnerability of FR systems tends to increase with face-verification accuracy. Using several publicly available PA datasets, we show that DNN-based FR systems compensate for variability between bona fide and PA samples, and tend to score them similarly, which makes such FR systems extremely vulnerable to PAs. Experiments show the vulnerability of the studied DNN-based FR systems to be consistently higher than 90%, and often higher than 98%.

show abstract

“…The MOBIO database, developed by Marcel et al (2012), is a unique audio-visual corpus, as it was captured almost exclusively using mobile devices. It is composed of over 61 h of recordings of 150 speakers.…”

Section: Review Of Audio-visual Corporamentioning

confidence: 99%

An audio-visual corpus for multimodal automatic speech recognition

et al. 2017

View full text Add to dashboard Cite

A review of available audio-visual speech corpora and a description of a new multimodal corpus of English speech recordings is provided. The new corpus containing 31 hours of recordings was created specifically to assist audio-visual speech recognition systems (AVSR) development. The database related to the corpus includes high-resolution, high-framerate stereoscopic video streams from RGB cameras, depth imaging stream utilizing Time-of-Flight camera accompanied by audio recorded using both: a microphone array and a microphone built in a mobile computer. For the purpose of applications related to AVSR systems training, every utterance was manually labeled, resulting in label files added to the corpus repository. Owing to the inclusion of recordings made in noisy conditions the elaborated corpus can also be used for testing robustness of speech recognition systems in the presence of acoustic background noise. The process of building the corpus, including the recording, labeling and post-processing phases is described in the paper. Results achieved with the developed audio-visual automatic speech recognition (ASR) engine trained and tested with the material contained in the corpus are presented and discussed together with comparative test results employing a state-of-the-art/commercial ASR engine. In order to demonstrate the practical use of the corpus it is made available for the public use.

show abstract

Bi-Modal Person Recognition on a Mobile Phone: Using Mobile Phone Data

Cited by 189 publications

References 14 publications

Discrimination Between Native and Non-Native Speech Using Visual Features Only

Discrimination Between Native and Non-Native Speech Using Visual Features Only

Deeply vulnerable: a study of the robustness of face recognition to presentation attacks

An audio-visual corpus for multimodal automatic speech recognition

Contact Info

Product

Resources

About