Osteoarthritis (OA) damages the articular cartilage of the knee and is a severe degenerative joint condition. Currently, OA diagnosis is carried out by symptom analysis and progressive evaluation of the radiographs, although the method is subjective. This work presents a novel computer aided diagnostic system to diagnose the severity of the disease in its early stages. The proposed method comprises of stages that include image pre-processing, extraction of features based on discrete wavelet decomposition, histogram, GLCM and texture features, and classification by stacked up-RBM Deep Belief Networks (Hybrid DBN) finally. The performance of HDBN is improved by optimizing the hyper parameters with the Salp Swarm Optimization Algorithm (SSA). Experiments conducted on the Kaggle database consisting of 7503 images demonstrated an overall accuracy of 99.45% with 100% sensitivity and specificity. The impact on isolated and combined feature contributions is also analyzed using 10-fold cross validation (CV).The robustness of the algorithm is tested on degraded images of salt-pepper, Gaussian, and Poisson noise, which proved the effectiveness of the method. Comparison of the method proposed with similar techniques is done by employing different optimization algorithms and classifiers.