In order to automatically recognize abnormal actions in physical education teaching in intelligent monitoring system, a CNN and HMM based action recognition model is proposed. In this scheme, the camera and computer are used to capture and measure the target, and the preliminary features of the image acquired from the data are extracted. Then, hybrid CNN-HMM is introduced as the key technology and main method of behavior recognition, and principal component analysis is also used to reduce the dimension of the extracted feature parameters. Finally, the test set is input into the trained classifier for action recognition, which provides an auxiliary reference for teaching actions. The experimental results show that the system can accurately quantify the human movements in sports teaching, and the recognition accuracy of complex databases with different scenes is better than that of similar methods.