Unsupervised Learning of Functional Categories in Video Scenes

Turek, Matthew W.; Hoogs, Anthony; Collins, Roderic

doi:10.1007/978-3-642-15552-9_48

Cited by 34 publications

(37 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other words, we directly represent objects in terms of function/affordances, without taking the visual appearance into account. This is similar in spirit to recent work [17], [16], [18], [19] in which scene regions with certain affordances, or function, are detected by observing (real or simulated) humans interacting with them.…”

Section: Introductionsupporting

confidence: 83%

Functional object descriptors for human activity modeling

Pieropan¹,

Ek²

2013

2013 IEEE International Conference on Robotics and Automation

View full text Add to dashboard Cite

Abstract-The ability to learn from human demonstration is essential for robots in human environments. The activity models that the robot builds from observation must take both the human motion and the objects involved into account. Object models designed for this purpose should reflect the role of the object in the activity -its function, or affordances. The main contribution of this paper is to represent object directly in terms of their interaction with human hands, rather than in terms of appearance. This enables the direct representation of object affordances/function, while being robust to intraclass differences in appearance. Object hypotheses are first extracted from a video sequence as tracks of associated image segments. The object hypotheses are encoded as strings, where the vocabulary corresponds to different types of interaction with human hands. The similarity between two such object descriptors can be measured using a string kernel. Experiments show these functional descriptors to capture differences and similarities in object affordances/function that are not represented by appearance.

show abstract

Section: Introductionsupporting

confidence: 83%

Functional object descriptors for human activity modeling

Pieropan¹,

Ek²

2013

2013 IEEE International Conference on Robotics and Automation

View full text Add to dashboard Cite

show abstract

“…Turek et al [63,40] used a similar idea to identify the functional map of a scene. Other approaches like [27,19,42,36] showed the use of scene semantics to predict goals and paths for human navigation.…”

Section: Related Workmentioning

confidence: 99%

Social LSTM: Human Trajectory Prediction in Crowded Spaces

Alahi

Goel

Ramanathan

et al. 2016

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

2,612

2,594

View full text Add to dashboard Cite

Pedestrians follow different trajectories to avoid obstacles and accommodate fellow pedestrians. Any autonomous vehicle navigating such a scene should be able to foresee the future positions of pedestrians and accordingly adjust its path to avoid collisions. This problem of trajectory prediction can be viewed as a sequence generation task, where we are interested in predicting the future trajectory of people based on their past positions. Following the recent success of Recurrent Neural Network (RNN) models for sequence prediction tasks, we propose an LSTM model which can learn general human movement and predict their future trajectories. This is in contrast to traditional approaches which use hand-crafted functions such as Social forces. We demonstrate the performance of our method on several public datasets. Our model outperforms state-of-the-art methods on some of these datasets . We also analyze the trajectories predicted by our model to demonstrate the motion behaviour learned by our model.

show abstract

“…Kembhavi et al [50] developed a video understanding system to identify various scene elements, such as roads, sidewalks and bus stops, using probabilistic models within a Markov logic network framework. From a perspective of categorical object recognition to scene analysis, Turek et al [51] proposed an approach to analyze a video scene based on the behaviors of moving objects in and around them. (a) A scene; and (b) its corresponding activity heat map that shows an aggregation of trajectories in that scene over a period of one hour.…”

Section: Harvesting Scene Activity Informationmentioning

confidence: 99%

Walk This Way: Improving Pedestrian Agent-Based Models through Scene Activity Analysis

Crooks

Croitoru

et al. 2015

IJGI

View full text Add to dashboard Cite

Pedestrian movement is woven into the fabric of urban regions. With more people living in cities than ever before, there is an increased need to understand and model how pedestrians utilize and move through space for a variety of applications, ranging from urban planning and architecture to security. Pedestrian modeling has been traditionally faced with the challenge of collecting data to calibrate and validate such models of pedestrian movement. With the increased availability of mobility datasets from video surveillance and enhanced geolocation capabilities in consumer mobile devices we are now presented with the opportunity to change the way we build pedestrian models. Within this paper we explore the potential that such information offers for the improvement of agent-based pedestrian models. We introduce a Scene-and Activity-Aware Agent-Based Model (SA 2 -ABM), a method for harvesting scene activity information in the form of spatiotemporal trajectories, and incorporate this information into our models. In order to assess and evaluate the improvement offered by such information, we carry out a range of experiments using

show abstract

Unsupervised Learning of Functional Categories in Video Scenes

Cited by 34 publications

References 14 publications

Functional object descriptors for human activity modeling

Functional object descriptors for human activity modeling

Social LSTM: Human Trajectory Prediction in Crowded Spaces

Walk This Way: Improving Pedestrian Agent-Based Models through Scene Activity Analysis

Contact Info

Product

Resources

About