Using Transfer Learning, SVM, and Ensemble Classification to Classify Baby Cries Based on Their Spectrogram Images

Le, Lillian; Kabir, Abu Nadim M.H.; Ji, Chunyan; Basodi, Sunitha; Pan, Yi

doi:10.1109/massw.2019.00028

Cited by 22 publications

(6 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nonetheless, integrating other features from other domains that improve the linear separation ability will be further investigated in our future works. As mentioned before, it has been shown that extracting spectrogram features includes important information or characteristics in classifying infant crying signals [ 34 , 35 , 36 , 37 ]. Combining spectrogram features along with the prosodic and cepstral features will be one of our future works.…”

Section: Discussionmentioning

confidence: 99%

“…Furthermore, image domain features, such as the spectrogram which is a time-frequency image representation of an audio signal and includes both acoustic and prosodic information, can be used to distinguish between healthy and unhealthy infant cries. It has been widely shown that feeding spectrograms into machine learning algorithms also plays an important role in enhancing the classification of different infant crying signals [ 34 , 35 , 36 , 37 ]. It is, therefore, obvious that each domain contributes to the classification of infant crying signals, and thus the mechanism of generating a combined feature set that takes advantage of different domains deserves to be considered and investigated.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Newborn Cry-Based Diagnostic System to Distinguish between Sepsis and Respiratory Distress Syndrome Using Combined Acoustic Features

2022

View full text Add to dashboard Cite

Crying is the only means of communication for a newborn baby with its surrounding environment, but it also provides significant information about the newborn’s health, emotions, and needs. The cries of newborn babies have long been known as a biomarker for the diagnosis of pathologies. However, to the best of our knowledge, exploring the discrimination of two pathology groups by means of cry signals is unprecedented. Therefore, this study aimed to identify septic newborns with Neonatal Respiratory Distress Syndrome (RDS) by employing the Machine Learning (ML) methods of Multilayer Perceptron (MLP) and Support Vector Machine (SVM). Furthermore, the cry signal was analyzed from the following two different perspectives: 1) the musical perspective by studying the spectral feature set of Harmonic Ratio (HR), and 2) the speech processing perspective using the short-term feature set of Gammatone Frequency Cepstral Coefficients (GFCCs). In order to assess the role of employing features from both short-term and spectral modalities in distinguishing the two pathology groups, they were fused in one feature set named the combined features. The hyperparameters (HPs) of the implemented ML approaches were fine-tuned to fit each experiment. Finally, by normalizing and fusing the features originating from the two modalities, the overall performance of the proposed design was improved across all evaluation measures, achieving accuracies of 92.49% and 95.3% by the MLP and SVM classifiers, respectively. The MLP classifier was outperformed in terms of all evaluation measures presented in this study, except for the Area Under Curve of Receiver Operator Characteristics (AUC-ROC), which signifies the ability of the proposed design in class separation. The achieved results highlighted the role of combining features from different levels and modalities for a more powerful analysis of the cry signals, as well as including a neural network (NN)-based classifier. Consequently, attaining a 95.3% accuracy for the separation of two entangled pathology groups of RDS and sepsis elucidated the promising potential for further studies with larger datasets and more pathology groups.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Newborn Cry-Based Diagnostic System to Distinguish between Sepsis and Respiratory Distress Syndrome Using Combined Acoustic Features

2022

View full text Add to dashboard Cite

show abstract

“…Finally, the performance was evaluated by classification methods, such as SVM and decision tree (DT). Le et al (2019) used spectrogram images of the infant cry audio and trained them with several methods to detect abnormalities that arise in the first few months of an infant's life and could cause a permanent critical problem if not treated at an early stage. The methods used in this study were transfer learning, SVM, and ensemble learning.…”

Section: Related Workmentioning

confidence: 99%

Machine learning-based infant crying interpretation

Hammoud,

Getahun,

Baldycheva

et al. 2024

Front. Artif. Intell.

View full text Add to dashboard Cite

Crying is an inevitable character trait that occurs throughout the growth of infants, under conditions where the caregiver may have difficulty interpreting the underlying cause of the cry. Crying can be treated as an audio signal that carries a message about the infant's state, such as discomfort, hunger, and sickness. The primary infant caregiver requires traditional ways of understanding these feelings. Failing to understand them correctly can cause severe problems. Several methods attempt to solve this problem; however, proper audio feature representation and classifiers are necessary for better results. This study uses time-, frequency-, and time-frequency-domain feature representations to gain in-depth information from the data. The time-domain features include zero-crossing rate (ZCR) and root mean square (RMS), the frequency-domain feature includes the Mel-spectrogram, and the time-frequency-domain feature includes Mel-frequency cepstral coefficients (MFCCs). Moreover, time-series imaging algorithms are applied to transform 20 MFCC features into images using different algorithms: Gramian angular difference fields, Gramian angular summation fields, Markov transition fields, recurrence plots, and RGB GAF. Then, these features are provided to different machine learning classifiers, such as decision tree, random forest, K nearest neighbors, and bagging. The use of MFCCs, ZCR, and RMS as features achieved high performance, outperforming state of the art (SOTA). Optimal parameters are found via the grid search method using 10-fold cross-validation. Our MFCC-based random forest (RF) classifier approach achieved an accuracy of 96.39%, outperforming SOTA, the scalogram-based shuffleNet classifier, which had an accuracy of 95.17%.

show abstract

“…Many fields of research utilized audio for classification. Interesting research was done on crying baby sounds to classify their present state as hungry, unwell, deaf, asphyxia, or normal in [23]. The goal of this study was to identify problems in their earliest stages by processing the babies' crying sounds since babies are unable to express themselves verbally.…”

Section: Utilizing Audio Signals For Classification Problemsmentioning

confidence: 99%

LimitAccess: on-device TinyML based robust speech recognition and age classification

Maayah

Abunada

Al-Janahi

et al. 2023

Discov Artif Intell

View full text Add to dashboard Cite

Automakers from Honda to Lamborghini are incorporating voice interaction technology into their vehicles to improve the user experience and offer value-added services. Speech recognition systems are a key component of smart cars, enhancing convenience and safety for drivers and passengers. In the future, safety-critical features may rely on speech recognition, but this raises concerns about children accessing such services. To address this issue, the LimitAccess system is proposed, which uses TinyML for age classification and helps parents limit children’s access to critical speech recognition services. This study employs a lite convolutional neural network (CNN) model for two different reasons: First, CNN showed superior accuracy compared to other audio classification models for age classification problems. Second, the lite model will be integrated into a microcontroller to meet its limited resource requirements. To train and evaluate our model, we created a dataset that included child and adult voices of the keyword “open”. The system approach categorizes voices into age groups (child, adult) and then utilizes that categorization to grant access to a car. The robustness of the model was enhanced by adding a new class (recordings) to the dataset, which enabled our system to detect replay and synthetic voice attacks. If an adult voice is detected, access to start the car will be granted. However, if a child’s voice or a recording is detected, the system will display a warning message that educates the child about the dangers and consequences of the improper use of a car. Arduino Nano 33 BLE sensing was our embedded device of choice for integrating our trained, optimized model. Our system achieved an overall F1 score of 87.7% and 85.89% accuracy. LimitAccess detected replay and synthetic voice attacks with an 88% F1 score.

show abstract

Using Transfer Learning, SVM, and Ensemble Classification to Classify Baby Cries Based on Their Spectrogram Images

Cited by 22 publications

References 4 publications

Newborn Cry-Based Diagnostic System to Distinguish between Sepsis and Respiratory Distress Syndrome Using Combined Acoustic Features

Newborn Cry-Based Diagnostic System to Distinguish between Sepsis and Respiratory Distress Syndrome Using Combined Acoustic Features

Machine learning-based infant crying interpretation

LimitAccess: on-device TinyML based robust speech recognition and age classification

Contact Info

Product

Resources

About