In the view of escalating global threat in security, it is imperative to have an automated detection system that can pick up suspicious patterns of human movement in physical environments. It can give a forewarning before a planned attack happens or an ultimate security is breached. In the past, significant research on the Intrusion Detection was established, but limited to virtual environments like computer networks and operating systems. In this paper, we proposed a general security model for detecting suspicious patterns in physical environment. Suspicious patterns are subtle and we showed that they can be detected via an experiment.