An Efficient Dense and Scale-Invariant Spatio-Temporal Interest Point Detector

Willems, Gert; Tuytelaars, Tinne; Gool, Luc Van

doi:10.1007/978-3-540-88688-4_48

Cited by 696 publications

(551 citation statements)

References 14 publications

Supporting

Mentioning

540

Contrasting

Unclassified

Order By: Relevance

“…Some of the most popular descriptors have been based on the gradient of the appearance information (Schuldt et al 2004;Laptev and Perez 2007), spatio-temporal extensions to SIFT and SURF descriptors (Scovanner et al 2007;Willems et al 2008) and 2D motion information (Dalal et al 2006;Messing et al 2009). However, there has been little previous work on feature descriptors including depth information (which is generally encoded directly at the holistic level, with the aid of user masks).…”

Section: Related Workmentioning

confidence: 99%

“…This can be done at various stages of the pipeline, but we first look at interest point extraction, in order to detect more salient features, and discount irrelevant detections. The extended algorithms discussed in this section are based on the Harris Corners work by Laptev and Lindeberg (2003), the Hessian points algorithm by Willems et al (2008) and the Separable Filters technique by Dollar et al (2005). For a comparison of the original 2D interest point detection schemes (without the proposed depth-aware extensions), see the survey paper by Tuytelaars and Mikolajczyk (2008).…”

Section: Interest Point Detectionmentioning

confidence: 99%

See 1 more Smart Citation

Hollywood 3D: What are the Best 3D Features for Action Recognition?

2016

View full text Add to dashboard Cite

Action recognition "in the wild" is extremely challenging, particularly when complex 3D actions are projected down to the image plane, losing a great deal of information. The recent growth of 3D data in broadcast content and commercial depth sensors, makes it possible to overcome this. However, there is little work examining the best way to exploit this new modality. In this paper we introduce the Hollywood 3D benchmark, which is the first dataset containing "in the wild" action footage including 3D data. This dataset consists of 650 stereo video clips across 14 action classes, taken from Hollywood movies. We provide stereo calibrations and depth reconstructions for each clip. We also provide an action recognition pipeline, and propose a number of specialised depth-aware techniques including five interest point detectors and three feature descriptors. Extensive tests allow evaluation of different appearance and depth encoding schemes. Our novel techniques exploiting this depth allow us to reach performance levels more than triple those of the best baseline algorithm using only appearance information. The benchmark data, code and calibrations are all made available to the community.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Interest Point Detectionmentioning

confidence: 99%

Hollywood 3D: What are the Best 3D Features for Action Recognition?

2016

View full text Add to dashboard Cite

show abstract

“…A cal-ibrated camera moves freely in two scenarios: multiple objects with and without repetition. To evaluate the performance of our system, a recognition is deemed valid if J > 0.5, as proposed in [19]. In the first scenario, five different objects are present in the scene, while the second scenario envisages four instances of the same object and two other objects.…”

Section: Multiple Object Recognitionmentioning

confidence: 99%

3D Object Recognition and Pose Estimation for Multiple Objects Using Multi-Prioritized RANSAC and Model Updating

Fenzi

Dragon

Leal-Taixé

et al. 2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We present a feature-based framework that combines spatial feature clustering, guided sampling for pose generation, and model updating for 3D object recognition and pose estimation. Existing methods fails in case of repeated patterns or multiple instances of the same object, as they rely only on feature discriminability for matching and on the estimator capabilities for outlier rejection. We propose to spatially separate the features before matching to create smaller clusters containing the object. Then, hypothesis generation is guided by exploiting cues collected off-and on-line, such as feature repeatability, 3D geometric constraints, and feature occurrence frequency. Finally, while previous methods overload the model with synthetic features for wide baseline matching, we claim that continuously updating the model representation is a lighter yet reliable strategy. The evaluation of our algorithm on challenging video sequences shows the improvement provided by our contribution.

show abstract

“…The BoF approach can also be extended to action recognition tasks. The most direct way to do so is to build a histogram for each video, where the features are extracted from groups of frames rather than from a single image (see, e.g., [21][22][23]). In the case of surgical gesture recognition, we extract Space-Time Interest Points (STIP) [21] from each video surgeme.…”

Section: Classification Using Bag Of Spatio-temporal Featuresmentioning

confidence: 99%

Surgical Gesture Classification from Video Data

Haro

Zappella

Vidal

2012

Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012

View full text Add to dashboard Cite

Abstract. Much of the existing work on automatic classification of gestures and skill in robotic surgery is based on kinematic and dynamic cues, such as time to completion, speed, forces, torque, or robot trajectories. In this paper we show that in a typical surgical training setup, video data can be equally discriminative. To that end, we propose and evaluate three approaches to surgical gesture classification from video. In the first one, we model each video clip from each surgical gesture as the output of a linear dynamical system (LDS) and use metrics in the space of LDSs to classify new video clips. In the second one, we use spatio-temporal features extracted from each video clip to learn a dictionary of spatio-temporal words and use a bag-of-features (BoF) approach to classify new video clips. In the third approach, we use multiple kernel learning to combine the LDS and BoF approaches. Our experiments show that methods based on video data perform equally well as the state-of-the-art approaches based on kinematic data.

show abstract

An Efficient Dense and Scale-Invariant Spatio-Temporal Interest Point Detector

Cited by 696 publications

References 14 publications

Hollywood 3D: What are the Best 3D Features for Action Recognition?

Hollywood 3D: What are the Best 3D Features for Action Recognition?

3D Object Recognition and Pose Estimation for Multiple Objects Using Multi-Prioritized RANSAC and Model Updating

Surgical Gesture Classification from Video Data

Contact Info

Product

Resources

About