Nowadays we have seen that the study of facial expression recognition became most important. In humans, the way of interaction will possible through text and voice. Apart from that, facial expressions play an important role in smooth communication among individuals. We can recognize the emotion of a human, based on facial expressions. First of all, facial areas were removed from the original images. Meanwhile, by using the different types of optimizers we have trained the convolutional neural network mainly focused on the face extraction. The accuracy of other methods like 2D PCA is 78.61%, Adaboost is 83.12%, SVM is 87.15%. In this paper, the method we proposed is softmax classifier by which we obtained an accuracy of 94.82% which is the highest among all. Here this softmax classifier we have used in this experiment, suggests the combination of face detection and emotion detection using CNN.