Tomato Health Monitoring System: Tomato Classification, Detection, and Counting System Based on YOLOv8 Model With Explainable MobileNet Models Using Grad-CAM++

Abstract

Fruits and vegetables (especially, tomatoes) healthy detection are important tasks for smart agriculture. Several works have been published in tomato detection, however, there is little research on using explainable AI to detect, classify and count tomato fruit status. In this work, we propose a Tomatoes Health Check System by evaluating MobileNet models based on the physiological tomato dataset. The MobileNet model is considered a model specifically designed to operate effectively on mobile devices and limited resources. Therefore, MobileNet models used in applying explainable artificial intelligence (XAI) problems must be considered. Our research conducts experiments to evaluate the accuracy of the MobileNets, MobileNetV2 and MobileNetV3 models based on the evaluation metrics; the highest accuracy of 96.69% belongs to the MobileNetV3 model. To evaluate the reliability of the model used, we use Grad-CAM++ to explain and evaluate reliability based on the Intersection over the Union parameter, resulting in MobileNetV2 being the model with the highest value with 100.00%(δ=0), 100.00% (δ=0.25) and 98.89% (δ=0.5). Our research conducted an evaluation experiment using the Simple online and real-time tracking algorithm to combine the YOLOv8 and MobileNetV2 algorithms to detect, classify and count the number of tomatoes according to physiology using video. Finally, we use the research results to build an application system.