Multi-resolution speech analysis for automatic speech recognition using deep neural networks: Experiments on TIMIT

Abstract

Speech Analysis for Automatic Speech Recognition (ASR) systems typically starts with a Short-Time Fourier Transform (STFT) that implies selecting a fixed point in the time-frequency resolution trade-off. This approach, combined with a Mel-frequency scaled filterbank and a Discrete Cosine Transform give rise to the Mel-Frequency Cepstral Coefficients (MFCC), which have been the most common speech features in speech processing for the last decades. These features were particularly well suited for the previous Hidden Markov Models/Gaussian Mixture Models (HMM/GMM) state of the art in ASR. In particular they produced highly uncorrelated features of small dimensionality (typically 13 coefficients plus deltas and double deltas), which was very convenient for diagonal covariance GMMs, for dealing with the curse of dimensionality and for the limited computing resources of a decade ago. Currently most ASR systems use Deep Neural Networks (DNN) instead of the GMMs for modeling the acoustic features, which provides more flexibility regarding the definition of the features. In particular, acoustic features can be highly correlated and can be much larger in size because the DNNs are very powerful at processing high-dimensionality inputs. Also, the computing hardware has reached a level of evolution that makes computational cost in speech processing a less relevant issue. In this context we have decided to revisit the problem of the time-frequency resolution in speech analysis, and in particular to check if multi-resolution speech analysis (both in time and frequency) can be helpful in improving acoustic modeling using DNNs. Our experiments start with several Kaldi baseline system for the well known TIMIT corpus and modify them by adding multi-resolution speech representations by concatenating different spectra computed using different time-frequency resolutions and different post-processed and speaker-adapted features using different time-frequency resolutions. Our experiments show that using a multi-resolution speech representation tends to improve over results using the baseline single resolution speech representation, which seems to confirm our main hypothesis. However, results combining multi-resolution with the highly post-processed and speaker-adapted features, which provide the best results in Kaldi for TIMIT, yield only very modest improvements.

Towards Deep Object Detection Techniques for Phoneme Recognition

“…Likewise, we observe that the proposed YOLOv3 model is on a par with most of stat-of-the-art models such as [50,51,100] and outperforms many of cutting edge models such as [1], [48], [52], [53].…”

“…A DBN with multiple hidden layers was also proposed by the same authors in [47] and achieved a 20.7% PER on TIMIT. Recently, a DNN acoustic model for TIMIT phone recognition based on multi resolution speech representation proposed in [48] achieved the best PER of 18.25%. The performances of a feed forward DNN, time delay neural network (TDNN), and long short-term memory (LSTM) are explored in [44] for TIMIT phone recognition, where LSTMbased phone recognition achieved a PER of 15.02%.…”

Allosteric Regulation at the Crossroads of New Technologies: Multiscale Modeling, Networks, and Machine Learning

“…The remarkable rise of deep learning (DL) relying on the robust function approximations and representation properties of deep neural networks has provided us with new tools to automatically find compact low-dimensional representations (features) of high-dimensional data (LeCun et al, 2015 ). DL models have achieved outstanding predictive performance making dramatic breakthroughs in a wide range of applications, including automatic speech processing and image recognition (Toledano et al, 2018 ; Kim et al, 2019 ; Hey et al, 2020 ; Wu et al, 2020 ). In the words of Geoffrey Hinton who is the founder of DL technologies “Deep Learning is an algorithm which has no theoretical limitations on what it can learn; the more data you give and the more computational time you provide the better it is” (LeCun et al, 2015 ).…”

Speech Recognition of Isolated Words using a New Speech Database in Sylheti

“…iii) Classification is the process of mapping the feature vector of an input word into 1 out of N word classes of the considered vocabulary during testing. Some popularly used classifiers in ASR are Artificial Neural Network (ANN) [5], [10], [12], [13], Hidden Markov model (HMM) [14], [15], Dynamic Time Warping (DTW) [16], [17], Deep Neural Network [9], [47], [51], etc. The application of ANN in designing ASR system is still being used by researchers [5], [6], [19], [20], [21], [22], [23], [36], [40], [42] despite the developments in the field of deep neural network (DNN) in recent times.…”

“…In recording the speech utterances, the following hardware and software These speakers do not have any history of speech disorders. As there is no specific rule about the male-female proportion in construction of speech database, literatures[51],[56],[58] have considered various proportions like 60%-40%, 70%-30%, 65%-35%, etc. The speakers in this work are chosen from Sylheti speaking areas in the Karimganj district of the state of Assam and the Kailasahar and Kumarghat districts of the state of Tripura, India where they have been living since their childhood.…”