Athlete’s heart rate measurement has certain guiding significance for athlete training and competition intensity arrangement. At present, the accuracy and efficiency of the athlete’s heart rate measurement method cannot meet the actual training needs of athletes. In view of this, based on support vector machine, this research combines with improved algorithm to build athlete heart rate measurement model. Moreover, in this study, the denoising algorithm of multi-channel spectral matrix decomposition is used to eliminate the interference factors. The heart rate measurement algorithm based on support vector machine (Mix-SVM) proposed by this paper mainly includes the following parts: preprocessing, preliminary filtering of motion noise, sparse signal reconstruction model, spectral subtraction, and heart rate spectral peak tracking method based on SVM. In addition, in order to verify the effectiveness of the algorithm in this study, a control experiment is designed to verify the efficiency and accuracy of the algorithm proposed by this study. The research results show that the algorithm proposed by this paper has certain advantages in accuracy and efficiency.