HMM-Based Speech Segmentation: Improvements of Fully Automatic Approaches

Abstract

Speech segmentation refers to the problem of determining the phoneme boundaries from an acoustic recording of an utterance together with its orthographic transcription. This paper focuses on a particular case of Hidden Markov Model (HMM) based forced alignment in which the models are directly trained on the corpus to align. The obvious advantage of this technique is that it is applicable to any language or speaking style and does not require manually-aligned data. Through a systematic stepby-step study, the role played by various training parameters (e.g. models configuration, number of training iterations) on the alignment accuracy is assessed, with corpora varying in speaking style and language. Based on a detailed analysis of the errors commonly made by this technique, we also investigate the use of additional fully automatic strategies to improve the alignment. Beside the use of supplementary acoustic features, we explore two novel approaches: an initialization of the silence models based on a Voice Activity Detection (VAD) algorithm and the consideration of the forced alignment of the timereversed sound. The evaluation is carried out on 12 corpora of different sizes, languages (some being under-resourced) and speaking styles. It aims at providing a comprehensive study of the alignment accuracy achieved by the different versions of the speech segmentation algorithm depending on corpus-related specificities. While the baseline method is shown to reach good alignment rates with corpora as small as 2 minutes, we also emphasize the benefit of using a few seconds of bootstrapping data. Regarding improvement methods, our results show that the insertion of additional features outperforms both other strategies. The performance of VAD, however, is shown to be notably striking on very small corpora, correcting more than 60 % of the errors superior to 40 ms. Finally, the combination of the three improvement methods is also pointed out as providing the highest alignment rates, with very low variability across the corpora, regardless of their size. This combined technique is shown to outperform available speaker-independent models, improving the alignment rate by 8 to 10 % absolute.

Singing Voice Phoneme Segmentation by Hierarchically Inferring Syllable and Phoneme Onset Positions

“…The baseline is a 1-state monophone DNN/HSMM model. We use monophone model because our small dataset doesn't have enough phoneme instances for exploring the context-dependent triphones model, also Brognaux and Drugman [6] and Pakoci et al [10] argued that context-dependent model can't bring significant alignment improvement. It is convenient to apply 1-state model because each phoneme can be represented by a semi-Markovian state carrying a state occupancy time distribution.…”

“…MFCCs) to the HMM states. Brognaux and Drugman [6] explored the forced alignment on a small dataset using supplementary acoustic features and initializing the silence model by voice activity detection algorithm. To predict the confidence measure of the aligned word boundaries and to fine-tune their time positions, Serriére et al [7] explored an alignment postprocessing method using a deep neural network (DNN).…”

Speech Segmentation Using Dynamic Windows and Thresholds for Arabic and English Languages

“…These features are the short-term energy, zero crossing rate and the singularity exponents calculated in each point of signal. While (Brognaux and Thomas, 2016) focuses on a particular case of hidden Markov model (HMM)-based forced alignment in which the models are directly trained on the corpus to align. Kamper et al (2017) introduces an approximation to a recent Bayesian model that still has a clear objective function but improves efficiency by using hard clustering and segmentation rather than full Bayesian inference.…”

Rapid Development of TTS Corpora for Four South African Languages

“…Since the phone set and G2P components for each language needs to be developed before script preparation and given that it is possible to perform automatic phonetic alignment with as few as 20 utterances [9,10], it may be worthwhile attempting to develop a tool that can automatically flag potentially significant divergences in pronunciation for manual inspection once a prototypical speaker has been identified.…”