Recent breakthroughs in deep learning-based convolutional neural networks have significantly improved image categorization accuracy. Deep learning-based techniques for diagnosing illnesses from rice plant images have been created in this work, inspired by the realisation of CNNs in image classification. Smart monitoring technologies for the automatic identification of plant diseases are extremely beneficial to sustainable agriculture. Despite the fact that various mechanisms for plant disease categorization have been created in recent years, an inefficient technique based on evidence from picture samples is of concern for ground environments. In this study, an image processing technique for pre-processing and segmentation was used, as well as a multi-class convolutional neural network with transfer learning, to classify rice plant leaf diseases such as brown spot, hispa, leaf blast, and healthy class. The contaminated area was automatically separated from the healthy areas of the image using canny edge detection and k-means clustering, and the features were retrieved using the CNN model. In the experimental results, the CNN model without transfer learning is compared to the transfer learning model. VGGNet transfer learning is used to construct a multi-classification framework for each class of rice illness. The overall accuracy acquired by the CNN model without transfer learning is 92.14%, whereas the accuracy obtained by the transfer learning model is 94.80%.The current work demonstrates that the proposed technique is compelling and capable of recognizing rice plant illness for four classes.