Pneumonia has caused significant deaths worldwide, and it is a challenging task to detect many lung diseases such as like atelectasis, cardiomegaly, lung cancer, etc., often due to limited professional radiologists in hospital settings. In this paper, we develop a straightforward VGG-based model architecture with fewer layers. In addition, to tackle the inadequate contrast of chest X-ray images, which brings about ambiguous diagnosis, the Dynamic Histogram Enhancement technique is used to pre-process the images. The parameters of our model are reduced by 97.51% compared to VGG-16, 85.86% compared to Res-50, 83.94% compared to Xception, 51.92% compared to DenseNet121, but increased MobileNet by 4%. However, the proposed model’s performance (accuracy: 96.068%, AUC: 0.99107 with a 95% confidence interval of [0.984, 0.996], precision: 94.408%, recall: 90.823%, F1 score: 92.851%) is superior to the models mentioned above (VGG-16: accuracy, 94.359%, AUC: 0.98928; Res-50: accuracy, 92.821%, AUC, 0.98780; Xception: accuracy, 96.068%, AUC, 0.99623; DenseNet121: accuracy, 87.350%, AUC, 0.99347; MobileNet: accuracy, 95.473%, AUC, 0.99531). The original Pneumonia Classification Dataset in Kaggle is split into three sub-sets, training, validation and test sets randomly at ratios of 70%, 10% and 20%. The model’s performance in pneumonia detection shows that the proposed VGG-based model could effectively classify normal and abnormal X-rays in practice, hence reducing the burden of radiologists.