Learning patterns of activity using real-time tracking

Abstract

ÐOur goal is to develop a visual monitoring system that passively observes moving objects in a site and learns patterns of activity from those observations. For extended sites, the system will require multiple cameras. Thus, key elements of the system are motion tracking, camera coordination, activity classification, and event detection. In this paper, we focus on motion tracking and show how one can use observed motion to learn patterns of activity in a site. Motion segmentation is based on an adaptive background subtraction method that models each pixel as a mixture of Gaussians and uses an on-line approximation to update the model. The Gaussian distributions are then evaluated to determine which are most likely to result from a background process. This yields a stable, real-time outdoor tracker that reliably deals with lighting changes, repetitive motions from clutter, and long-term scene changes. While a tracking system is unaware of the identity of any object it tracks, the identity remains the same for the entire tracking sequence. Our system leverages this information by accumulating joint co-occurrences of the representations within a sequence. These joint cooccurrence statistics are then used to create a hierarchical binary-tree classification of the representations. This method is useful for classifying sequences, as well as individual instances of activities in a site. Index TermsÐReal-time visual tracking, adaptive background estimation, activity modeling, co-occurrence clustering, object recognition, video surveillance and monitoring (VSAM).

Reducing Bidirectional Pedestrian Conflict Based on Lane Formation Phenomenon in Subway Corridors

“…(LK) [27]. To verify the accuracy of automotive identifying arithmetic, trial experiments were conducted before the formal experiments, showing that almost all pedestrians participating in the experiment could be detected and tracked.…”

Socially intelligent surveillance and monitoring: Analysing social dimensions of physical space

“…In almost every surveillance system published in literature, the main goal is that of promptly identifying threatening behaviours in an automatic way. Assuming that the meaning of behavior has been grounded, the underlying, subtle, problem is that the meaning of "threatening" is actually unspecified, and reduced (too) often to that of "abnormal" or "unexpected" [62,18]. This translates in having complex techniques that simply collect a statistics of trajectories, and whenever a different trajectory does hold, it is labelled as abnormal and considered as potentially threatening.…”

“…In this case, it becomes feasible to model and retrieve events like "pedestrian crosses the road on the crosswalk", considering a priori manual definition of a set of pre-determined normal and abnormal events. In another spirit, similar systems encapsulate the capability of learning in an unsupervised way what is usual in a given scenario and what is not, considering sufficient statistics of trajectories [29,62]. In the meanwhile, multi-camera and multi-object tracking methods are becoming able to track multiple objects across far locations, captured by sensors with non overlapping camera views [64,65].…”

“…1c). The third evolution step added a further higher level to the previous architecture, devoted to reasoning about the trajectories of the foreground objects [52,29,62]. This level allows the user to specify events of interest as particular trajectories, drawn by a precise class of objects (see Fig.…”

Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components

“…For example, vehicle driver assistance systems must observe street scenes [1,2] or the driving person [3] in real-time. Similarly, gesture recognition systems or visual surveillance tools are most useful when they are able to process video material onthe-fly, such as [4,5,6]. Real-time visual processing is also required for continuous environmental mapping and localization for robots [7] or for wearable computers [8].…”