DIVA (Directions Into of Articulators) model is a kind of self-adaptive neutral network model which controls movements of a simulated vocal tract in order to produce words, syllables or phonemes. However, there exist poor classification ability, out of consideration of overlap and other deficiencies among multiple modeling primitives in current Hidden Markov(HMM) training algorithm. It impacts speech recognition rate of the model. Therefore, this paper proposes a hybrid model HMM/PNN, which is to use Predictive Neural Network (PNN) in ANN(Artificial neutral network) to calculate station posterior distribution of Hidden Markov Model. The acoustic model of DIVA is reconstructed through extracting acoustic parameter, choosing modeling unit and other methods. The simulations show that after training and learning the pronunciation of compound vowel by using new HMM/PNN model, there s not huge difference between the waveform of the acquired speech and that of real person, in addition, the recognition rate is also improved. All these verify the effectiveness and accuracy of this method.