In the recent era facial image processing is gaining more importance and the face detection from image or from video have number of applications which are video surveillance, entertainment, security, multimedia, communication, Ubiquitous computing etc. Various research work are carried out for face detection and processing which includes detection, tracking of the face, estimation of pose, clustering the detected faces etc. Although significant advances have been made, the performance of face detection systems provi de satisfactory under controlled environment & may get degraded with some challenging scenario such as in real time video face detection and processing. There are many real-time applications where human face serves as identity and these application are time bound so time for detection of face from image or video and the further processing is very essential, thus here our goal is to discuss the face detection system overview and to review various human skin colors based approaches and Haar feature based approach for better detection performance. Detected faces tagging and clustering is essential in some cases, so for such further processing time factor plays important role. Some of the recent approaches to improve detection speed such as using Graphical Processing Unit are discussed and providing future directions in this area.