Towards capturing fine phonetic variation in speech using articulatory features

Abstract

The ultimate goal of our research is to develop a computational model of human speech recognition that is able to capture the effects of fine-grained acoustic variation on speech recognition behaviour. As part of this work we are investigating automatic feature classifiers that are able to create reliable and accurate transcriptions of the articulatory behaviour encoded in the acoustic speech signal. In the experiments reported here, we analysed the classification results from support vector machines (SVMs) and multilayer perceptrons (MLPs). MLPs have been widely and successfully used for the task of multi-value articulatory feature classification, while (to the best of our knowledge) SVMs have not. This paper compares the performance of the two classifiers and analyses the results in order to better understand the articulatory representations. It was found that the SVMs outperformed the MLPs for five out of the seven articulatory feature classes we investigated while using only 8.8–44.2% of the training material used for training the MLPs. The structure in the misclassifications of the SVMs and MLPs suggested that there might be a mismatch between the characteristics of the classification systems and the characteristics of the description of the AF values themselves. The analyses showed that some of the misclassified features are inherently confusable given the acoustic space. We concluded that in order to come to a feature set that can be used for a reliable and accurate automatic description of the speech signal; it could be beneficial to move away from quantised representations

Rethinking classification results based on read speech, or: why improvements do not always transfer to other speaking styles

“…Also King et al (2007) mentions SVMs as a powerful classification technique for binary tasks. Then, Scharenborg et al (2007) showed that SVMs even compare favorably to ANNs on the task of multi-level articulatory-acoustic feature classification, which is also the task at hand in the current work. We therefore use SVMs for our investigations.…”

“…In one line of research, exemplified by Frankel et al (2007b), King and Taylor (2000), Manjunath and Sreenivasa Rao (2016) and Scharenborg et al (2007), attempts are made to cover a full set of features with a single multi-value classifier (with seven classes). In the second line, exemplified by Niyogi et al (1999), Pruthi and Espy-Wilson (2007), and Schutte and Glass (2005), 1 3 research concentrates on finding an optimal set of acoustic parameters for building a detector for one specific manner feature for, e.g., vowel nasalization (e.g., Pruthi and EspyWilson 2007), nasal manner (e.g., Chen 2000; Pruthi and Espy-Wilson 2004), or stops (e.g., Abdelatti Ali et al 2001;Niyogi et al 1999).…”

“…For building AF classifiers, different statistical classifiers have been used, such as artificial neural networks (ANNs) (e.g., King and Taylor 2000;Kirchhoff 1998;Scharenborg et al 2007;Siniscalchi et al 2008) and FFNNs (Manjunath and Sreenivasa Rao 2016), HMMs (e.g., Kirchhoff 1999;Manjunath and Sreenivasa Rao 2016), linear dynamic models (Frankel 2003), k nearest neighbor (k-NNs) (Naess et al 2011) and dynamic Bayesian Networks (e.g., Pernkopf et al 2009;Frankel et al 2007b;Jyothi 2013). Niyogi et al (1999), Pruthi and Espy-Wilson (2007), Yoon et al (2010), Scharenborg et al (2007), and Schutte and Glass (2005) used support vector machines (SVMs) for the classification of articulatory-acoustic features, among other reasons because these show good generalization from a small amount of high-dimensional training data.…”

“…Niyogi et al (1999), Pruthi and Espy-Wilson (2007), Yoon et al (2010), Scharenborg et al (2007), and Schutte and Glass (2005) used support vector machines (SVMs) for the classification of articulatory-acoustic features, among other reasons because these show good generalization from a small amount of high-dimensional training data. Juneja (2004) developed SVM-based landmark detectors for classifying binary place and voicing features in TIMIT, while Niyogi and Sondhi (2001) used SVMs to detect stop consonants in TIMIT.…”

“…Research on AF classification has focused on finding the ideal set of acoustic parameters for building multi-value or binary classifiers (e.g., Salomon et al 2004;King and Taylor 2000;Niyogi et al 1999;Scharenborg et al 2007) or the ideal statistical classification method (e.g., a comparison of ANNs with SVMs by Chang et al 2005). For an overview of different methods used for AF classification see King et al (2007).…”