A novel objective function for improved phoneme recognition using time-delay neural networks

Abstract

Single-speaker and multispeaker recognition results are presented for the voice-stop consonants /b,d,g/ using time-delay neural networks (TDNNs) with a number of enhancements, including a new objective function for training these networks. The new objective function, called the classification figure of merit (CFM), differs markedly from the traditional mean-squared-error (MSE) objective function and the related cross entropy (CE) objective function. Where the MSE and CE objective functions seek to minimize the difference between each output node and its ideal activation, the CFM function seeks to maximize the difference between the output activation of the node representing incorrect classifications. A simple arbitration mechanism is used with all three objective functions to achieve a median 30% reduction in the number of misclassifications when compared to TDNNs trained with the traditional MSE back-propagation objective function alone.

Contour Plots of Objective Functions for Feed-Forward Neural Networks

“…In this case, using the max rule for classification, 1 y should be greater than 2 y for correct classification. That is, the region 2 1 y y > corresponds to a correct classification referred to as a "hit" [7]. On the contrary, the region 2 1 y y < corresponds to an incorrect classification referred to as a "miss".…”

“…Using CFM, learning focuses most heavily on the reduction of misclassifications rather than reducing the difference between the output node value and its target output. Thus, CFM is defined by [7] ( ) ( ) Let's consider a training of FNN based on MSE. When outliers are presented to the FNN, the difference between the output node value and its desired value is very large and this causes a heavy updating of parameters of FNN.…”

“…Parameters of FNNs are updated to decrease the error function during training [5]. For performance improvement of MSE, additive noise in the desired value is adopted as [6] ( ), The classification figure of merit (CFM) objective function has three essential features that distinguish it from the traditional MSE [7]. Firstly, it has not the desired value but the index of output node representing the correct classification outcome.…”

A Family-based Association Study of DIO2 and children mental retardation in the Qinba region of China

“…We used the time delay neural networks (TDNN) method 24 and the TFSEARCH program 25 to analyze the 50 bp sequence surrounding each SNP, and identify gain or loss of physical transcription factor-binding sites and promoter sites generated by the nucleotide exchange corresponding to alternative SNP alleles. Two analytical softwares were used to verify whether each SNP of the DIO2 gene has putative regulatory activity that may affect its transcription and expression.…”

Classification-based objective functions

“…The classification figure-of-merit (CFM) objective function was introduced in Hampshire II (1990) for learning classification problems when it was shown that SSE and CE errors are not necessarily correlated with classification accuracy. CFM separates the values of network outputs by as large a range as possible such that error minimization is monotonic with increasing classification accuracy.…”