Exploration of Acoustic and Lexical Cues for the INTERSPEECH 2020 Computational Paralinguistic Challenge

Yang, Ziqing; An, Zifan; Fan, Zehao; Jing, Chengye; Cao, Huiluo

doi:10.21437/interspeech.2020-2999

Cited by 5 publications

(5 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yang et al. [65] explore Fisher Vector (FV) encoders [66] , a widely used computer vision technique that uses a Gaussian Mixture Model (GMM), trained to model the distribution of a set of low-level features. The Fisher Vector encapsulates the first-order and second-order gradients of the features with respect to the GMM model.…”

Section: Challenge Results and Contributionsmentioning

confidence: 99%

“…The method followed by [65] is generic as it depends on generic audio features, namely several representations of the ComParE feature set combined with FV, which can be applied for audio processing in general.…”

Section: Challenge Results and Contributionsmentioning

confidence: 99%

“… [63] 75.4 21.8 27.4 RNN for phoneme recognition 11 Yang et al. [65] 75.1 18.5 31.3 Fisher Vectors on ComParE and MFCC, early and late fusions 13 Ristea and 74.6 29.2 21.6 Cycle-consistent GANs Ionescu [59] for augmentation 18 Schuller et al. [38] 71.8 16.5 40.0 Deep Spectrum , BoWA, ComParE , auDeep 19 Illium et al.…”

Section: Challenge Results and Contributionsmentioning

confidence: 99%

“… [58] utilise ensembles of a particular model with different snapshots during training, either by training several times using different optimisation algorithms or just taking the same model during one training at different steps. In [54] , [59] , [61] , [65] , ensembles are used as well. In fact, the approaches by Illium et al.…”

Section: Discussionmentioning

confidence: 99%

See 3 more Smart Citations

Face mask recognition from audio: The MASC database and an overview on the mask challenge

Mohamed

Nessiem

Batliner

et al. 2022

Pattern Recognition

View full text Add to dashboard Cite

The sudden outbreak of COVID-19 has resulted in tough challenges for the field of biometrics due to its spread via physical contact, and the regulations of wearing face masks. Given these constraints, voice biometrics can offer a suitable contact-less biometric solution; they can benefit from models that classify whether a speaker is wearing a mask or not. This article reviews the Mask Sub-Challenge (MSC) of the INTERSPEECH 2020 COMputational PARalinguistics challengE (ComParE), which focused on the following classification task: Given an audio chunk of a speaker, classify whether the speaker is wearing a mask or not. First, we report the collection of the Mask Augsburg Speech Corpus (MASC) and the baseline approaches used to solve the problem, achieving a performance of Unweighted Average Recall (UAR). We then summarise the methodologies explored in the submitted and accepted papers that mainly used two common patterns: (i) phonetic-based audio features, or (ii) spectrogram representations of audio combined with Convolutional Neural Networks (CNNs) typically used in image processing. Most approaches enhance their models by adapting ensembles of different models and attempting to increase the size of the training data using various techniques. We review and discuss the results of the participants of this sub-challenge, where the winner scored a UAR of . Moreover, we present the results of fusing the approaches, leading to a UAR of . Finally, we present a smartphone app that can be used as a proof of concept demonstration to detect in real-time whether users are wearing a face mask; we also benchmark the run-time of the best models.

show abstract

Section: Challenge Results and Contributionsmentioning

confidence: 99%

Section: Challenge Results and Contributionsmentioning

confidence: 99%

Section: Challenge Results and Contributionsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Face mask recognition from audio: The MASC database and an overview on the mask challenge

Mohamed

Nessiem

Batliner

et al. 2022

Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…In total, 13 articles were categorized as reporting outcomes related to acoustic classification of face masks, in which the aim of the authors was to identify whether a speaker was wearing a face mask from a set of acoustic features or algorithms. Of these, seven articles were published in the Proceedings of Interspeech 2020 as a part of the Computational Paralinguistics Mask Sub-Challenge [64][65][66][67][68][69][70]. The goal of this challenge was for authors to identify whether a speaker was wearing a surgical mask or not, by using machine learning techniques to identify acoustic features from audio recordings from the Mask Augsburg Speech Corpus (MASC) [71].…”

Section: Acoustic Classification Of Masksmentioning

confidence: 99%

Acoustic and perceptual impact of face masks on speech: A scoping review

Badh,

Knowles

2023

PLoS ONE

View full text Add to dashboard Cite

During the COVID-19 pandemic, personal protective equipment such as facial masks and coverings were mandated all over the globe to protect against the virus. Although the primary aim of wearing face masks is to protect against viral transmission, they pose a potential burden on communication. The purpose of this scoping review was to identify the state of the evidence of the effect of facial coverings on acoustic and perceptual speech outcomes. The scoping review followed the framework created by Arksey & O’Malley (2005) and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews guidelines (PRISMA-ScR; Tricco et al., 2018). The search was completed in May 2021 across the following databases: PubMed, EMBASE, PsycINFO, Web of Science, and Google Scholar. A total of 3,846 records were retrieved from the database search. Following the removal of duplicates, 3,479 remained for the title/abstract screen and 149 were selected for the full-text review. Of these, 52 were included in the final review and relevant data were extracted. The 52 articles included in the final review consisted of; 11 studied perceptual outcomes only, 16 studied acoustic outcomes only, and 14 studied both perceptual and acoustic outcomes. 13 of these investigated acoustic features that could be used for mask classification. Although the findings varied from article to article, many trends stood out. Many articles revealed that face masks act as a low pass filter, dampening sounds at higher frequencies; however, the frequency range and the degree of attenuation varied based on face mask type. All but five articles that reported on perceptual outcomes showed a common trend that wearing a face mask was associated with poorer speech intelligibility. The findings of the scoping review provided evidence that facial coverings negatively impacted speech intelligibility, which is likely due to a combination of auditory and visual cue degradation. Due to the continued prevalence of mask use, how facial coverings affect a wider variety of speaker populations, such as those with communication impairments, and strategies for overcoming communication challenges should be explored.

show abstract

ViT-LSTM synergy: a multi-feature approach for speaker identification and mask detection

Nassif,

Shahin,

Bader

et al. 2024

Neural Comput & Applic

View full text Add to dashboard Cite

Exploration of Acoustic and Lexical Cues for the INTERSPEECH 2020 Computational Paralinguistic Challenge

Cited by 5 publications

References 6 publications

Face mask recognition from audio: The MASC database and an overview on the mask challenge

Face mask recognition from audio: The MASC database and an overview on the mask challenge

Acoustic and perceptual impact of face masks on speech: A scoping review

ViT-LSTM synergy: a multi-feature approach for speaker identification and mask detection

Contact Info

Product

Resources

About