Multi-modal voice pathology detection architecture based on deep and handcrafted feature fusion

Omeroglu, Asli Nur; Mohammed, Hussein Mahmood Abdo; Oral, Emin Argun

doi:10.1016/j.jestch.2022.101148

Cited by 16 publications

(6 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, a two-stage approach is used in which expert-derived voice features, most commonly the MFCC, 6,12,15,16 are calculated from the raw voice data and used to predict vocal pathology. 28 Second, a single sustained vowel recording (e.g., selected vowel /a/ samples) is used as the initial raw data input. 15,16,29 The proposed framework offers several advantages.…”

Section: Discussionmentioning

confidence: 99%

“…First, a two‐stage approach is used in which expert‐derived voice features, most commonly the MFCC, 6 , 12 , 15 , 16 are calculated from the raw voice data and used to predict vocal pathology. 28 Second, a single sustained vowel recording (e.g., selected vowel /a/ samples) is used as the initial raw data input. 15 , 16 , 29 Even when the model is trained with different sustained vowel recordings (e.g., selected vowel /a/, /i/, or /u/ samples), only an individual vowel recording is input to generate a prediction of vocal pathology.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

End‐to‐end deep learning classification of vocal pathology using stacked vowels

Liu,

Hodges,

et al. 2023

Laryngoscope Investig Oto

View full text Add to dashboard Cite

ObjectivesAdvances in artificial intelligence (AI) technology have increased the feasibility of classifying voice disorders using voice recordings as a screening tool. This work develops upon previous models that take in single vowel recordings by analyzing multiple vowel recordings simultaneously to enhance prediction of vocal pathology.MethodsVoice samples from the Saarbruecken Voice Database, including three sustained vowels (/a/, /i/, /u/) from 687 healthy human participants and 334 dysphonic patients, were used to train 1‐dimensional convolutional neural network models for multiclass classification of healthy, hyperfunctional dysphonia, and laryngitis voice recordings. Three models were trained: (1) a baseline model that analyzed individual vowels in isolation, (2) a stacked vowel model that analyzed three vowels (/a/, /i/, /u/) in the neutral pitch simultaneously, and (3) a stacked pitch model that analyzed the /a/ vowel in three pitches (low, neutral, and high) simultaneously.ResultsFor multiclass classification of healthy, hyperfunctional dysphonia, and laryngitis voice recordings, the stacked vowel model demonstrated higher performance compared with the baseline and stacked pitch models (F1 score 0.81 vs. 0.77 and 0.78, respectively). Specifically, the stacked vowel model achieved higher performance for class‐specific classification of hyperfunctional dysphonia voice samples compared with the baseline and stacked pitch models (F1 score 0.56 vs. 0.49 and 0.50, respectively).ConclusionsThis study demonstrates the feasibility and potential of analyzing multiple sustained vowel recordings simultaneously to improve AI‐driven screening and classification of vocal pathology. The stacked vowel model architecture in particular offers promise to enhance such an approach.Lay SummaryAI analysis of multiple vowel recordings can improve classification of voice pathologies compared with models using a single sustained vowel and offer a strategy to enhance AI‐driven screening of voice disorders.Level of Evidence3

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

End‐to‐end deep learning classification of vocal pathology using stacked vowels

Liu,

Hodges,

et al. 2023

Laryngoscope Investig Oto

View full text Add to dashboard Cite

show abstract

“…The study used 10-fold cross-validation and accuracies of 87.11% and 86.52% for CNN and RNN, respectively. A combination of features from EGG and speech from sustained vowel \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $/a/$\end{document} has been considered for distinguishing normal and pathological voices [35] , [36] . Other studies considered stacked autoencoder [37] and LSTM-based autoencoder [38] using sustained vowel \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $/a/$\end{document} and continuous speech samples, respectively.…”

Section: Introductionmentioning

confidence: 99%

“…The study used 10-fold crossvalidation and accuracies of 87.11% and 86.52% for CNN and RNN, respectively. A combination of features from EGG and speech from sustained vowel /a/ has been considered for distinguishing normal and pathological voices [35,36]. Other studies considered stacked autoencoder [37] and LSTM-based autoencoder [38] using sustained vowel /a/ and continuous speech samples, respectively.…”

Section: Introductionmentioning

confidence: 99%

Multitask and Transfer Learning Approach for Joint Classification and Severity Estimation of Dysphonia

Aziz,

Dávid

2024

IEEE J. Transl. Eng. Health Med.

View full text Add to dashboard Cite

Objective: Despite speech being the primary communication medium, it carries valuable information about a speaker’s health, emotions, and identity. Various conditions can affect the vocal organs, leading to speech difficulties. Extensive research has been conducted by voice clinicians and academia in speech analysis. Previous approaches primarily focused on one particular task, such as differentiating between normal and dysphonic speech, classifying different voice disorders, or estimating the severity of voice disorders. Methods and procedures: This study proposes an approach that combines transfer learning and multitask learning (MTL) to simultaneously perform dysphonia classification and severity estimation. Both tasks use a shared representation; network is learned from these shared features. We employed five computer vision models and changed their architecture to support multitask learning. Additionally, we conducted binary ‘healthy vs. dysphonia’ and multiclass ‘healthy vs. organic and functional dysphonia’ classification using multitask learning, with the speaker’s sex as an auxiliary task. Results: The proposed method achieved improved performance across all classification metrics compared to single-task learning (STL), which only performs classification or severity estimation. Specifically, the model achieved F1 scores of 93% and 90% in MTL and STL, respectively. Moreover, we observed considerable improvements in both classification tasks by evaluating beta values associated with the weight assigned to the sex-predicting auxiliary task. MTL achieved an accuracy of 77% compared to the STL score of 73.2%. However, the performance of severity estimation in MTL was comparable to STL. Conclusion: Our goal is to improve how voice pathologists and clinicians understand patients’ conditions, make it easier to track their progress, and enhance the monitoring of vocal quality and treatment procedures. Clinical and Translational Impact Statement: By integrating both classification and severity estimation of dysphonia using multitask learning, we aim to enable clinicians to gain a better understanding of the patient’s situation, effectively monitor their progress and voice quality.

show abstract

“…Since they do not rely on an individual's judgment, these assessment techniques are objective. Plus, they're simple to implement because several online recoding apps make it possible to access speech recordings from anywhere [7]. Hence, in order to reliably differentiate between healthy individuals and those with voice pathologies, several studies have devised vocal processing methods to ascertain which aspects of vocal pathology, when combined with a method, can effectively detect voice pathology automatically within a single framework [8].…”

Section: Introductionmentioning

confidence: 99%

AROA based Pre-trained Model of Convolutional Neural Network for Voice Pathology Detection and Classification

Nachimuthu

et al. 2024

JMC

View full text Add to dashboard Cite

With the demand for better, more user-friendly HMIs, voice recognition systems have risen in prominence in recent years. The use of computer-assisted vocal pathology categorization tools allows for the accurate detection of voice pathology diseases. By using these methods, vocal disorders may be diagnosed early on and treated accordingly. An effective Deep Learning-based tool for feature extraction-based vocal pathology identification is the goal of this project. This research presents the results of using EfficientNet, a pre-trained Convolutional Neural Network (CNN), on a speech pathology dataset in order to achieve the highest possible classification accuracy. An Artificial Rabbit Optimization Algorithm (AROA)-tuned set of parameters complements the model's mobNet building elements, which include a linear stack of divisible convolution and max-pooling layers activated by Swish. In order to make the suggested approach applicable to a broad variety of voice disorder problems, this study also suggests a unique training method along with several training methodologies. One speech database, the Saarbrücken voice database (SVD), has been used to test the proposed technology. Using up to 96% accuracy, the experimental findings demonstrate that the suggested CNN approach is capable of detecting speech pathologies. The suggested method demonstrates great potential for use in real-world clinical settings, where it may provide accurate classifications in as little as three seconds and expedite automated diagnosis and treatment.

show abstract

Multi-modal voice pathology detection architecture based on deep and handcrafted feature fusion

Cited by 16 publications

References 36 publications

End‐to‐end deep learning classification of vocal pathology using stacked vowels

End‐to‐end deep learning classification of vocal pathology using stacked vowels

Multitask and Transfer Learning Approach for Joint Classification and Severity Estimation of Dysphonia

AROA based Pre-trained Model of Convolutional Neural Network for Voice Pathology Detection and Classification

Contact Info

Product

Resources

About