Automatic gender classification from facial image has become an attractive research area in the field of machine learning. Various methods have already been proposed for gender recognition in both controlled and uncontrolled situations. Problem arises in uncontrolled situation when there are high rate of noises, lack of illumination etc. To mitigate the problems, we have proposed a framework where we applied a pre-processing to enhance the images using Bilateral Histogram Equalization (BHEP) algorithm and applied the proposed framework in LFW, Adience and color FERET dataset yielding 94.29%, 84.86% and 98.30% accuracies. Confusion matrix, Precision, Recall, F-measure, True Positive Rate (TPR), True Negative Rate (TNR) etc. also shows that our proposed method performs better than the existing state of the arts.