Spatiotemporal fusion target detection and location based on depth optical flow and YOLOv3

Abstract

The accurate detection and location of a human foot moving target is the prerequisite for a small robot to realize short-range robust tracking of human moving targets. An F-YOLOv3 (FlowNet2-YOLOv3) algorithm based on deep optical flow and neural network spatiotemporal information fusion that can accurately detect and locate foot moving targets is proposed. The algorithm extracts the motion information of the human foot target in the time domain and the position information of the human foot target in the space domain through the FlowNet2 and YOLOv3 networks, respectively, for foot detection. Then, according to the established fusion strategy, the temporal and spatial information of the foot target extracted by the two networks is fused to realize the foot target location. Five groups of human walking videos of typical scenes are used to simulate the detection and location of foot targets. The results show that the target detection precision of the F-YOLOv3 fusion algorithm is 85% in the scene of a human turning and foot occlusion and 90% in the scene with drastic illumination change. Compared with the YOLOv3 algorithm, the average mAP value obtained by F-YOLOv3 is increased by 3.89%. Finally, under the robot operating system, the F-YOLOv3 algorithm is applied to the turtlebot robot, and foot detection and positioning in the process of human walking are successfully completed.