Abstract. Visual tracking is a hot topic in the field of computer vision, this paper firstly describes the classification and application of visual tracking and highlights the advantages of object tracking based on infrared image by simple comparison. Combined with the research of this technology, tracking technology is divided into five categories, which are named region-based tracking, model-based tracking, activity-based tracking, feature-based tracking, and filteringbased tracking respectively. Finally, based on the characteristics of infrared images, this paper expounds the difficulties in the visual tracking of infrared images and prospects the research prospect of infrared image visual tracking.