Mispronunciation Detection Using Deep Convolutional Neural Network Features and Transfer Learning-Based Model for Arabic Phonemes

Nazir, Faria; Majeed, Muhammad Nadeem; Ghazanfar, Mustansar Ali; Maqsood, Muazzam

doi:10.1109/access.2019.2912648

Cited by 40 publications

(19 citation statements)

References 29 publications

(41 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our work, we used deep learning and machine learning methods for the classification process. In order to input data into our classifiers, deep features were extracted [37], [38] from our speech signal dataset. We used deep learning convolution model Alexnet for extracting deep features from the pc-Gita dataset.…”

Section: B Feature Extraction: Deep Features Using Alexnet Modelmentioning

confidence: 99%

A Spectrogram-Based Deep Feature Assisted Computer-Aided Diagnostic System for Parkinson’s Disease

et al. 2020

Self Cite

View full text Add to dashboard Cite

Parkinson's disease is a neural degenerative disease. It slowly progresses from mild to severe stage, resulting in the degeneration of dopamine cells of neurons. Due to the deficiency of dopamine cells in the brain, it leads to a motor (tremor, slowness, impaired posture) and non-motor (speech, olfactory) defects in the body. Early detection of Parkinson's disease is a difficult chore as the symptoms of disease appear overtime. However, different diagnostic systems have contributed towards disease detection by considering gait, tremor and speech characteristics. Recent work has shown that speech impairments can be considered as a possible predictor for Parkinson's disease classification and remains an open research area. The speech signals show major differences and variations for Parkinson patients as compared to normal human beings. Therefore, variation in speech should be modeled using acoustic features to identify these variations. In this research, we propose three methods-the first method employs a transfer learning-based approach using spectrograms of speech recordings, the second method evaluates deep features extracted from speech spectrograms using machine learning classifiers and the third method evaluates simple acoustic feature of recordings using machine learning classifiers. The proposed frameworks are evaluated on a Spanish dataset pc-Gita. The results show that the second framework shows promising results with deep features. The highest 99.7% accuracy on vowel \o\ and read text is observed using a multilayer perceptron. Whereas 99.1% accuracy observed on vowel \i\ deep features using random forest. The deep feature-based method performs better as compared to simple acoustic features and transfer learning approaches. The proposed methodology outperforms the existing techniques on the pc-Gita dataset for Parkinson's disease detection.

show abstract

Section: B Feature Extraction: Deep Features Using Alexnet Modelmentioning

confidence: 99%

A Spectrogram-Based Deep Feature Assisted Computer-Aided Diagnostic System for Parkinson’s Disease

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…For phone-level recognition systems, the word error rate is called a PER [76]. In our experiments, we used the HResults analysis tool from the HTK toolkit [77] to calculate PER, which is computed by (3).…”

Section: ) Phone Error Rate (Per)mentioning

confidence: 99%

“…Phoneme recognition plays a dominant part in many applications such as speech recognition [1], speaker recognition [2], and pronunciation error detection and correction [3]. With the success of deep learning techniques for computer vision, many studies have been conducted on speech processing tasks by converting speech signals to a visual representation such as spectrogram [4].…”

Section: Introductionmentioning

confidence: 99%

Towards Deep Object Detection Techniques for Phoneme Recognition

et al. 2020

View full text Add to dashboard Cite

The use of cutting edge object detection techniques to build an accurate phoneme sequence recognition system for English and Arabic languages is investigated in this study. Recently, numerous techniques have been proposed for object detection in daily life applications using deep learning. In this paper, we propose the use of object detection techniques in speech processing tasks. We selected two state-of-the-art object detectors, namely YOLO and CenterNet, based on a trade-off between detection accuracy and speed. We tackled the problem of phoneme sequence recognition using three systems: the domain transfer learning system (DTS) from image to speech, intra-language transfer leaning system (IaTS) between speech corpora within the same language (English to English), and inter-language transfer learning system (IeTS) between speech corpora from dissimilar languages (English to Arabic). For English phoneme recognition, the Texas Instruments/Massachusetts Institute of Technology (TIMIT) corpus is used to evaluate the performance of the proposed systems. Our IaTS based on the CenterNet detector achieves the best results using the test core set of TIMIT with 15.89% phone error rate (PER). For Arabic phoneme recognition, the best performance, with 7.58% PER, was achieved using the CenterNet. These results show the effectiveness of using object detection techniques in phoneme recognition tasks. Furthermore, based on the findings of this study, speech processing tasks may be treated as object detection tasks. INDEX TERMS CenterNet, object detection, phoneme recognition, transfer learning, YOLO.

show abstract

“…Moreover, this approach is also common for solving regression problems. The performance of the DL model improves when the frequency of the data increases [55]. In this study, we use a DL model based on the Convolutional Neural Network with the standard-setting.…”

Section: Support Vector Regressionmentioning

confidence: 99%

An Efficient Deep Learning Based Model to Predict Interest Rate Using Twitter Sentiment

et al. 2020

View full text Add to dashboard Cite

In macroeconomics, decision making is highly sensitive and significantly influences the financial and business world, where the interest rate is a crucial factor. In addition, the interest rate is used by the governments to manage the monetary policy. There is a need to design an efficient algorithm for interest rate prediction. The analysis of the social media sentiment impact on financial decision making is also an open research area. In this study, we deploy a deep learning model for the accurate forecasting of the interest rate for the UK, Turkey, China, Hong Kong, and Mexico. For this purpose, daily data of the interest rate and exchange rate covering the period from Jan 2010 to Oct 2019 is used for all the mentioned countries. We also incorporate the input of the twitter sentiments of six mega-events, namely the US election 2012, Mexican election 2012, Gaza under attack 2014, Hong Kong protest 2014, Refugee Welcome 2015, and Brexit 2016. Our results provide evidence that the error of the deep learning model significantly decreases when event sentiment is incorporated. A notable improvement has been observed in the case of the Hong Kong interest rate, i.e., a 266% decline in the error after incorporating event sentiments as an input in the deep learning model.

show abstract

Mispronunciation Detection Using Deep Convolutional Neural Network Features and Transfer Learning-Based Model for Arabic Phonemes

Cited by 40 publications

References 29 publications

A Spectrogram-Based Deep Feature Assisted Computer-Aided Diagnostic System for Parkinson’s Disease

A Spectrogram-Based Deep Feature Assisted Computer-Aided Diagnostic System for Parkinson’s Disease

Towards Deep Object Detection Techniques for Phoneme Recognition

An Efficient Deep Learning Based Model to Predict Interest Rate Using Twitter Sentiment

Contact Info

Product

Resources

About