Real‐time inspection and removal of individual Fusarium head blight (FHB) infected corn grains from the processing lines has been a challenging issue due to the bulk handling and smaller kernel size. In this study, four different variants (small(s), medium(m), nano(n), and large(l)) of You Only Look Once (YOLO) version 5 object detection technique were trained for the detection of Fusarium infection in a moving monolayer of touching and non‐touching corn grains. The YOLOv5 object detection models were evaluated for their performance in detecting FHB infection in individual corn grains. A heterogeneous dataset containing images and video frames of healthy and FHB infected corn grains in different illuminations was utilized. The mean average precision calculated at Intersection over Union threshold of .5 (mAP@50) was 99%, 98%, 95%, and 96% for YOLOv5‐s, YOLOv5‐m, YOLOv5‐n, and YOLOv5‐l models, respectively. The detection speed in videos was 3.9, 1.6, 9.8, and .8 frames per second for YOLOv5‐s, m, n, and l models, respectively. For non‐touching grains, all four variants of the YOLOv5 model showed 100% precision, but for touching grains, all variants showed false negatives in detection of FHB infection, especially on overlapping kernels. The recall values were found to be 98%, 99%, 96%, and 97% for YOLOv5‐s, m, n, and l models, respectively. The best combination of mAP, detection speed, and lower false negatives was achieved by the YOLOv5‐m model. YOLOv5‐m has the potential for use in real‐time detection of Fusarium infection in corn grains apart from lag time in videos.Practical ApplicationThe developed video analysis technique based on YOLOv5 object detection method will be beneficial for the accurate identification of Fusarium infected corn grains in bulk handling facilities. The individual FHB infected grains could be detected on processing lines and could be used for real‐time inspection replacing the random sampling techniques currently used, thereby preventing the entry of Fusarium mycotoxins in the food chain. For non touching corn grains, all the YOLOv5 model variants showed a 100% precision in identifying the healthy and FHB infected grains. For touching grains, YOLOv5‐m model showed the best combination of mAP, detection speed, and lower false negatives proving appropriate for inspection on moving conveyor belts. The nano model with the lightweight architecture installed in portable devices can be used for immediate detection of FHB infection without lag time.