Conditional random fields for activity recognition

Abstract

To act intelligently in the presence of others, robots must use information from their sensors to recognize the behaviors and activities of the other agents in their environment. Robots must map from low-level, difficult to interpret data, such as position information extracted from video, to abstract states, in particular, the activities of the other agents. In this thesis, we explore how to bridge the gap from noisy, continuous observations about the world to high-level, discrete activity labels for robots in the environment.We contribute the use of conditional random fields (CRFs) for activity recognition in multirobot domains. We explore the appropriateness of CRFs with an empirical comparison to hidden Markov models. We elucidate the properties of CRFs that make them well suited to the activity recognition, namely discriminative training, the ability to robustly incorporate rich features of the observations, and their nature as conditional models, with a variety of synthetic and real robot data.Accurate activity recognition requires complex and rich features of the observations. We choose the most informative features from a large set of candidates using feature selection. We adapt two feature selection algorithms, grafting and 1 regularization, to conditional random fields. We also investigate a third feature selection algorithm, which was originally proposed for CRFs in a natural language processing domain, in an activity recognition context. In particular, we focus on scaling feature selection to very large sets of candidate features that we define succinctly using a rich relational feature specification language.The reduced feature sets that we discover via feature selection enable efficient, real-time inference. However, feature selection and training for conditional random fields is computationally expensive. We adapt an M-estimator, introduced by Jeon and Lin for log-density estimation in ANOVA models, for fast, approximate parameter estimation in CRFs. We provided an in depth, empirical evaluation of the properties of the M-estimator and then we introduce a new, efficient feature selection algorithm for CRFs based around M-estimation to identify the most important features.

Learning to Recognize Human Activities from Soft Labeled Data

“…1) How detrimental is hard labeling: Fig. 4 shows the performance of two baseline approaches ( [5] in dashed lines and the state-of-the-art approach [19] in dotted lines). We can see that both of the baselines drop dramatically with the decreasing confidence level.…”

“…A variety of graphical models have been proposed in the literature. Vail et al [19] model the activities in a linear-chain structure, where activity nodes of the temporal segments are inter-connected. Koppula et al [10] model both activities and objects affordance as random variables.…”

“…We compare our approach against four baseline methods. The first baseline uses a linear-chain CRF for activity recognition [19]. The second baseline is adopted from [10], which assumes multiple hypotheses of video segmentation.…”

“…1). This topic has been widely studied in both robotics communities [3,5,10,14] and other fields [15,19,20]. Most of the work uses a dataset where labels are hard assigned regardless of uncertainty.…”

A probabilistic logic programming event calculus

“…To overcome such limitations, these models have been extended in order to support more complex relations. Examples of such extensions include representing interactions involving multiple domain objects (Brand et al 1997;Gong and Xiang 2003;Vail et al 2007;Wu et al 2007), capturing long-term dependencies between states (Hongeng and Nevatia 2003), as well as describing a hierarchical composition of activities (Liao et al 2007;Natarajan and Nevatia 2007). However, the lack of a formal representation language makes the definition of complex LTA complicated and the integration of domain background knowledge very hard.…”