Detecting People Carrying Objects Utilizing Lagrangian Dynamics

Abstract

The availability of dense motion information in computer vision domain allows for the effective application of Lagrangian techniques that have their origin in fluid flow analysis and dynamical systems theory. A well established technique that has been proven to be useful in image-based crowd analysis are Finite Time Lyapunov Exponents (FTLE). Based on this, we present a method to detect people carrying object and describe a methodology how to apply established flow field methods onto the problem of describing individuals. Further, we reinterpret Lagrangian features in relation to the underlying motion process and show their applicability towards the appearance modeling of pedestrians. This definition allows to increase performance of state-of-the-art methods and is shown to be robust under varying parameter settings and different optical flow extraction approaches.

A Lagrangian framework for video analytics

“…Despite capturing large scale motion events it has also been used to evaluate motion patterns in microscopic images of cilia organelles by Lukens et al [7]. FTLE based descriptors of motion features have been show to improve the performance of state of the art techniques for visual detection tasks as shown by Senst et al [8]. In close relation to this, Brox et al [9] presented a particlebased segmentation based on geometric trajectory clustering.…”

“…The FTLE value is obtained by considering the time normalized logarithm of the spatial flow map derivatives. More details on this formalism and its implementation are presented by Senst et al [8]. Besides this predefined measures, any scalar descriptor available in the spatio-temporal context (e.g.…”

“…Since FTLE describes only motion boundaries, this has to be accomplished by artificially closing the resulting ridges using enclosed regions as segments. In contrast, combining forward and backward regions [12], [13], [8] allows to directly obtain coherent motion sets as enclosed regions. However, FTLE ridges in optical flow fields are created by a variety of different separation effects, and are prone to over-segmentation of complex motion fields.…”

A method of carried object detection based on key posture of silhouette

“…Tobias Senst et al use gray value information and present features by periodicity dependency descriptor [4]. A method which adopted the HOG descriptor directly on the FTLE field based on Lagrangian dynamics is also proposed by Tobias Senst [5]. A learningbased method is proposed in [6], the candidate image regions are from protrusion, color contrast and occlusion boundary cues.…”

Gait-Based Carried Object Detection Using Persistent Homology

“…Other cue used by authors is the periodicity in the moving of different body parts and their relationship through the time [2][3][4]. Approaches based on gait are an alternative to the localization approach when there are self-occlusions that prevent the object visualization or when the difference between the object's color and the clothes' color is very small to obtain correct segmentation [5].…”