Automatic segmentation of infant cry signals using hidden Markov models

Abstract

Automatic extraction of acoustic regions of interest from recordings captured in realistic clinical environments is a necessary preprocessing step in any cry analysis system. In this study, we propose a hidden Markov model (HMM) based audio segmentation method to identify the relevant acoustic parts of the cry signal (i.e., expiratory and inspiratory phases) from recordings made in natural environments with various interfering acoustic sources. We examine and optimize the performance of the system by using different audio features and HMM topologies. In particular, we propose using fundamental frequency and aperiodicity features. We also propose a method for adapting the segmentation system trained on acoustic material captured in a particular acoustic environment to a different acoustic environment by using feature normalization and semi-supervised learning (SSL). The performance of the system was evaluated by analyzing a total of 3 h and 10 min of audio material from 109 infants, captured in a variety of recording conditions in hospital wards and clinics. The proposed system yields frame-based accuracy up to 89.2%. We conclude that the proposed system offers a solution for automated segmentation of cry signals in cry analysis applications.

Video and audio processing in paediatrics: a review

“…HMM and Gaussian Mixture Model (GMM) methods were compared and GMM gave the best results with a classification error rate of 8.9% (Abou-Abbas et al, 2017b). A total accuracy of 89.2% was also reached with HMM in (Naithani et al, 2018).…”

“…Cry segmentation Simple Inverse Filter Tracking Orlandi et al 2012b Short Time Energy Dìaz et al 2012Orlandi et al 2012aManfredi et al 2018Várallyay 2006 Zero Crossing Rate Várallyay 2006Várallyay , 2007 Fourier Transform Várallyay 2007 Word reliability Yamamoto et al 2013 Classification K-Nearest Neighbor Reggiannini et al 2013 Hidden Markov Model Naithani et al 2018 Abou-Abbas et al 2015, 2017b Gaussian Mixture Model Abou-Abbas et al 2017b Logistic regression Lavner et al 2016Ferreti et al 2018 Convolutional Neural Networks Lavner et al2016Torres et al 2017Ferreti et al 2018 Cry classification Statistical analysis Grunau et al 1990Fuller 1991, Fuller and Conner 1995Johnston et al 1993Stevens et al 1994 Goberman Orlandi et al 2015O...…”

“…Hidden Markov Model (HMM) based approaches led to 83.79% of accuracy (Abou-Abbas et al, 2015) when six classes (expiratory phases, inspiratory phases, noise, adult speech, silence and beeps) were considered. Later, these results were improved by formulating the problem with three classes: expiratory phases, inspiratory phases and others (Abou-Abbas et al, 2017b) or residuals (Naithani et al, 2018) (a class regrouping all previous mentioned noisy sounds). HMM and Gaussian Mixture Model (GMM) methods were compared and GMM gave the best results with a classification error rate of 8.9% (Abou-Abbas et al, 2017b).…”

“…background noise) (Várallyay, 2006(Várallyay, , 2007Orlandi et al, 2012a;Díaz et al, 2012;Orlandi et al, 2012b;Reggiannini et al, 2013;Orlandi et al, 2013Orlandi et al, , 2015Orlandi et al, , 2016Manfredi et al, 2018) and ii) with high energy noise (e.g. medical device sounds, adults' voices) (Yamamoto et al, 2013;Abou-Abbas et al, 2015;Lavner et al, 2016;Abou-Abbas et al, 2017b;Naithani et al, 2018).…”

“…As mentioned in Section 3.2.2, cepstral coefficients (MFCCs and LPCCs) have also been extracted to describe audio signals (Lederman et al, 2008;Wahid et al, 2016;Pokorny et al, 2016;Abou-Abbas et al, 2017b;Naithani et al, 2018). MFCCs were traditionally computed in six steps: a) partitioning the signal into short frames, b) computing the power spectrum density, c) projecting the power spectra on mel-filter bank (simulation of human auditory perception) and sum the energy in each filter, d) taking the logarithm of all filter bank energies e) calculating the Discrete Cosinus Transform ( Time-varying frequency features Melody shapes have been identified by two methods based on the projection of F0 pattern into a grid: Five-Line Method (FLN) (fixed 5-lines grid from 330 Hz to 700 Hz) (Várallyay, 2007) and dodecagram (variable grid depending on the first F0 estimation of the cry) (Díaz et al, 2012).…”