A new classification algorithm of human action recognition of video content is suggested in our paper. It analyzes the variation of the content of video scenes or human action from video bi-modal features, in order to cognize content efficiently and precisely. This scheme is based on the pattern analysis of spatio-temporal slices and audio signature feature extracted from the video files. The Spatial-temporal Variation Histogram feature is firstly defined in our paper. It is applied to describe spatio-temporal variation analysis of human action or video scenes. The audio signature is also applied to identify the audio content by extracting unique signatures from a part of audio signal. The experiments show excellent performance of classification on the KTH dataset.