Action Recognition by Weakly-Supervised Discriminative Region Localization

Boyraz, Hakan; Masood, Syed Zain; Liu, Baoyuan; Tappen, Marshall F.; Foroosh, Hassan

doi:10.5244/c.28.111

Cited by 12 publications

(11 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Keeping these observations in mind, we propose a unique classifier that achieves a symmetric response with comparable precision and recall values while keeping both of them high. While our proposal is targeting the specific application area of image annotation [98]- [103], the proposed principle of symmetric classifier has potential applications in diverse areas of image processing and computer vision, such as self-localization [21], [45], [49], surveillance [43], [44], [48], action recognition [3], [5]- [8], [14], [85]- [87], [92]- [94], target localization and tracking [66], [67], [84], [89], shape description and object recognition [1], [16], [17], [117], image-based rendering [2], [9], [10], [88], image restoration [12], [39], [63], [78]- [83], and camera motion classification and quantification [4], [19]- [23], [46], [47], to name a few. The proposed annotation system employs multiple layers of sparse coding treating training images as predictors and the test image as target signal.…”

Section: Discussionmentioning

confidence: 99%

Designing a symmetric classifier for image annotation using multi-layer sparse coding

Tariq

Foroosh

2018

Image and Vision Computing

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

Designing a symmetric classifier for image annotation using multi-layer sparse coding

Tariq

Foroosh

2018

Image and Vision Computing

View full text Add to dashboard Cite

“…In order to avoid these time-consuming annotations, weakly supervised methods have been introduced recently for training action classifiers [21,22,23]. Boyraz et al…”

Section: Related Workmentioning

confidence: 99%

Automatic action annotation in weakly labeled videos

Sultani

Shah

2017

Computer Vision and Image Understanding

View full text Add to dashboard Cite

Manual spatio-temporal annotation of human action in videos is laborious, requires several annotators and contains human biases. In this paper, we present a weakly supervised approach to automatically obtain spatio-temporal annotations of an actor in action videos. We first obtain a large number of action proposals in each video. To capture a few most representative action proposals in each video and evade processing thousands of them, we rank them using optical flow and saliency in a 3D-MRF based framework and select a few proposals using MAP based proposal subset selection method. We demonstrate that this ranking preserves the high quality action proposals. Several such proposals are generated for each video of the same action. Our next challenge is to iteratively select one proposal from each video so that all proposals are globally consistent. We formulate this as Generalized Maximum Clique Graph problem using shape, global and fine grained similarity of proposals across the videos.The output of our method is the most action representative proposals from each video.Our method can also annotate multiple instances of the same action in a video. We have validated our approach on three challenging action datasets: UCF Sport, sub-JHMDB and THUMOS'13 and have obtained promising results compared to several baseline methods. Moreover, on UCF Sports, we demonstrate that action classifiers trained on these automatically obtained spatio-temporal annotations have comparable performance to the classifiers trained on ground truth annotation.

show abstract

“…The literature on human action recognition is primarily dominated by methods that rely on video data as input [8], [10]- [12], [15], [16], [33], [132]- [137], [143], [146]. Video data has the advantage of providing temporal information, which plays an important role in distinguishing different actions.…”

Section: Introductionmentioning

confidence: 99%

Still Image Action Recognition by Predicting Spatial-Temporal Pixel Evolution

Safaei

Foroosh

2019

2019 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

We propose a novel approach based on deep Convolutional Neural Networks (CNN) to recognize human actions in still images by predicting the future motion, and detecting the shape and location of the salient parts of the image. We make the following major contributions to this important area of research: (i) We use the predicted future motion in the static image (Walker et al., 2015) as a means of compensating for the missing temporal information, while using the saliency map to represent the the spatial information in the form of location and shape of what is predicted as significant. (ii) We cast action classification in static images as a domain adaptation problem by transfer learning. We first map the input static image to a new domain that we refer to as the Predicted Optical Flow-Saliency Map domain (POF-SM), and then fine-tune the layers of a deep CNN model trained on classifying the ImageNet dataset to perform action classification in the POF-SM domain. (iii) We tested our method on the popular Willow dataset. But unlike existing methods, we also tested on a more realistic and challenging dataset of over 2M still images that we collected and labeled by taking random frames from the UCF-101 video dataset. We call our dataset the UCF Still Image dataset or UCFSI-101 in short. Our results outperform the state of the art.

show abstract

Action Recognition by Weakly-Supervised Discriminative Region Localization

Cited by 12 publications

References 37 publications

Designing a symmetric classifier for image annotation using multi-layer sparse coding

Designing a symmetric classifier for image annotation using multi-layer sparse coding

Automatic action annotation in weakly labeled videos

Still Image Action Recognition by Predicting Spatial-Temporal Pixel Evolution

Contact Info

Product

Resources

About