Speech Recognition is a tremendous application from the history that is identification and conversion of spoken words into text. The performance and quality of work had increased a lot. This performance lead to the research work on Emotion Recognition based on the language spoken that is obtaining the kind of emotion from the spoken speech which is an application based on human-robot interactions. Emotions can be recognized in a better way using Speech processing, Artificial Intelligence techniques and linguistic semantics. Systems are given training in such a way to detect the emotions from the spoken utterances. This paper contains about the survey from the history to the present works that took place in the speech emotion recognition and also the experiment results. The survey contains about the works that took place from the by different scientists and their usage of different features, classifiers etc. The paper also holds three categories, one is different databases that are involved, second is what features are involved for representation of speech and third about the classification schemes. The survey also includes the conclusions of performances and limitations of current speech emotion recognition.