Nowadays, speech recognition applications are becoming increasingly effective. In the market, different interactive speech aware applications are obtainable. In this work, from the input speech signal by recognizing the content involves three stages such as the preprocessing, feature extraction and Multi Support Vector Machine (SVM). The signal is processed and noise free signal is produced by processing the signal and the features are extracted. For optimize these features different optimization algorithms are utilized. From this algorithm the optimal features such as peak signal frequency, Tri-spectral feature, and discrete wave transform (DWT) attain the APSO technique. These optimal features are given as the input of the multi SVM and the signal in testing process, the signal correctly recognize the text. From the results the optimization algorithm (APSO) obtains the 97.8% accuracy compared to the existing technique SVM linear kernel function.