SummaryFor controlling and managing the traffic and to help traffic surveillance, the vehicles classification is a matter of great importance. In the last few decades, vehicle classification systems based on pattern recognition have been utilized to enhance the efficiency for traffic monitoring systems. In the literature many deep learning networks are suggested for vehicle classification. Even though deep learning algorithms are fascinating and growing research area. However, there are several barriers that slow down its progress. The greatest factor that reduces the progress of deep learning systems is the quality of the image. The available vehicle image datasets are affected by noise, weather, and illumination variations. To overcome these issues, we suggest a robust deep learning system by combining bilateral filter individually with three different networks for the improvement of the robustness of vehicle classification in real‐time application. For validation the suggested networks are assessed on CompCars dataset. The study of literature has reported many Suggested CNN models, some of are discussed here. Lee et al. have employed SqueezeNet model and achieved accuracy of 0.963. Wang et al. have achieved accuracy of 0.989 through H‐squeezeNet model. By employing Faster‐RCNN Inception and SSD MobileNet‐v2 Giron et al. have achieved accuracy of 0.864. Zhang et al. have used Inception‐v3 and attained an accuracy of 0.974. In this article, the proposed CNN bilateral integrated models have accomplished the accuracies as 0.999, 0.997, and 0.988 for Inception‐v3, MobileNet‐v2, SqueezeNet networks, respectively. The results demonstrate the recommended techniques outperform the traditional deep learning networks.