In this review paper a number of studies and researches are surveyed, in order to assist the upcoming researchers, to know about the techniques available in the field of semantic based video retrieval. The video retrieval system is used for finding the users' desired video among a huge number of available videos on the Internet or database. This paper gives a general discussion on the overall process of the semantic video retrieval phases. In addition to its present a generic review of techniques that has been proposed to solve the semantic gap as the major scientific problem in semantic based video retrieval. The semantic gap is formed because of the difference between the low level features that are extracted from video content and user's perceptions of these features in a real world. The transformation of low level features of the video content into meaningful semantic concepts is a research topic that has received considerable attention in recent years.