Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi

Abstract

We present the Montreal Forced Aligner (MFA), a new opensource system for speech-text alignment. MFA is an update to the Prosodylab-Aligner, and maintains its key functionality of trainability on new data, as well as incorporating improved architecture (triphone acoustic models and speaker adaptation), and other features. MFA uses Kaldi instead of HTK, allowing MFA to be distributed as a stand-alone package, and to exploit parallel processing for computationally-intensive training and scaling to larger datasets. We evaluate MFA's performance on aligning word and phone boundaries in English conversational and laboratory speech, relative to human-annotated boundaries, focusing on the effects of aligner architecture and training on the data to be aligned. MFA performs well relative to two existing open-source aligners with simpler architecture (Prosodylab-Aligner and FAVE), and both its improved architecture and training on data to be aligned generally result in more accurate boundaries.

Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders

“…To measure the accessibility of phonetic information, we train linear phone classifiers using Mel-features, APC and Mockingjay representations from the LibriSpeech train-clean-360 subset. We obtain force-aligned phoneme sequences with the Montreal Forced Aligner [24], where there are 72 possible phone classes. Testing results on the LibriSpeech test-clean subset are presented in Figure 3.…”

Speech Model Pre-Training for End-to-End Spoken Language Understanding

“…Additionally, we find that using phonemes as intermediate targets speeds up word-level pre-training [37][38][39]. We use the Montreal Forced Aligner [40] to obtain wordand phoneme-level alignments for LibriSpeech, and we pretrain the model on the entire 960 hours of training data using these alignments 5,6 . Using force-aligned labels has the additional benefit of enabling pre-training using short, random crops rather than entire utterances, which reduces the computation and memory required to pre-train the model.…”

Models of Visually Grounded Speech Signal Pay Attention to Nouns: A Bilingual Experiment on English and Japanese

“…We forced aligned each spoken caption to its transcription (using the Montreal Forced Aligner [18] and Maus Forced Aligner [19] for English and Japanese respectively), resulting in alignments at word and phone level. We also tagged each dataset using TreeTagger [20] for English and KyTea [21] for Japanese.…”