With China’s booming economy, agricultural modernization has become an unstoppable trend. In order to solve the problem of real-time detection of multiple kinds of fruits, a real-time detection system of multiple kinds of fruits based on Raspberry Pi was developed. We trained the YOLOv5 network on a homemade multi-species fruit dataset. In the end, it not only realized the recognition of various types of fruits but also was able to output their location coordinates in the image. At the same time, we used the edge computing device Intel Neural Compute Stick 2 to realize the feed-forward inference operation on the neural network, so that the problem of low frame rate, when the Raspberry Pi performs realtime detection on a variety of fruits, is solved. In the real scenario, the real-time fruit detection model based on YOLOv5s network can effectively achieve the target detection of multiple types of fruits with 99.86% detection mAP, 99.93% testing accuracy and 10.37 FPS average real-time fruit detection frame rate. The system has many advantages, such as high accuracy, good robustness, fast calculation speed, etc. In addition, the embedded system can be further extended to realize the development of many intelligent products related to fruit recognition, such as intelligent fruit-sacking robots. Therefore, it can be concluded that the real-time detection system of multi-species fruits based on Raspberry Pi is helpful to improve the efficiency of farming, and the product can contribute to the modernization of agriculture in China.