Plan-view trajectory estimation with dense stereo background models

Abstract

In a known environment, objects may be tracked in multiple views using a set of background models. Stereo-based models can be illumination-invariant, but often have undefined values which inevitably lead to foreground classification errors. We derive dense stereo models for object tracking using long-term, extended dynamic-range imagery, and by detecting and interpolating uniform but unoccluded planar regions. Foreground points are detected quickly in new images using pruned disparity search. We adopt a "latesegmentation" strategy, using an integrated plan-view density representation. Foreground points are segmented into object regions only when a trajectory is finally estimated, using a dynamic programming-based method. Object entry and exit are optimally determined and are not restricted to special spatial zones.

Untethered gesture acquisition and recognition for virtual world manipulation

“…The disparity maps are grayscale images whose pixel intensities (disparities) are inversely proportional to the depth of the corresponding point. In our framework, the disparity maps are further processed to recover the approximate position of the user using background subtraction techniques [9]. Given the parameters of the stereo camera (focal lengths, baseline) a full 3D reconstruction of the user is computed and used as input for our articulated tracking algorithm.…”

Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera

“…In prior methods for tracking in planview maps, other researchers have used only occupancy data, and the person models are no more complex than Gaussians [3,6,10,15]. We have recently demonstrated the benefits of plan-view height data, as well as "template" person models that are constructed directly from, and matched directly to, the plan-view image data [14].…”

“…Plan-view occupancy maps similar to ours have been used in person tracking methods developed by other researchers [3,6,10,15], but height maps have not. Height maps preserve about as much 3D shape information as is possible in a 2D image, and therefore seem better suited than occupancy maps, which discard virtually all object shape information in the vertical dimension, for distinguishing people from each other and from other objects.…”

“…These models are allowed to adapt over time during tracking, to reflect changes in people's body configuration and visibility to the camera. While easy to construct and evolve, these models provide much richer descriptions of tracked people than simpler choices such as Gaussians or "blobs" used in prior planview tracking work [3,10,15], thereby enabling more accurate tracking of people through partial occlusions and close interactions. We also leverage these richer models to develop novel body pose and activity recognition techniques.…”

“…Unfortunately, the substantial noise, imprecise object contours, and large regions of low-confidence data typical of currently available real-time depth imagery (see Figure 3(b)) make application of standard image analysis and tracking methods to such imagery less reliable than is customary for color or grayscale video. In person detection and tracking contexts, some researchers have therefore found it useful to create "plan-view" projections of the input depth data, in which the 3D data inherent in a depth image is re-rendered as if viewed from an overhead, orthographic camera [3,6,10,15]. Because people typically do not overlap much in the dimension normal to the ground, plan-view projections of depth data allow people to be more easily separated and tracked than in the original "camera-view" depth images.…”