Key-Frame based Event Recognition in Unconstrained Videos using Temporal Features

Jana, Pulak Kumar; Bhaumik, Swarnabja; Mohanta, Partha Pratim

doi:10.1109/tensymp46218.2019.8971058

Cited by 6 publications

(2 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Table I, we tabulate the top-1 accuracy obtained by recent state-of-the-arts that employ 3D CNN or 2D CNN with LSTM [24], along with the input clip dimension (represented as #Channels × #Frames × FrameHeight × FrameWidth). One one hand, it can be concluded that pretrained 3D CNN architectures outperforms their relatively complex 2D counterparts combined with LSTM, signifying the potential of 3D CNNs in video classification.…”

Section: Comparison With State-of-the-artsmentioning

confidence: 99%

Unsupervised Action Localization Crop in Video Retargeting for 3D ConvNets

Jana¹,

Bhaumik²,

Mohanta³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Untrimmed videos on social media or those captured by robots and surveillance cameras are of varied aspect ratios. However, 3D CNNs usually require as input a squareshaped video, whose spatial dimension is smaller than the original. Random-or center-cropping may leave out the video's subject altogether. To address this, we propose an unsupervised video cropping approach by shaping this as a retargeting and video-to-video synthesis problem. The synthesized video maintains a 1:1 aspect ratio, is smaller in size and is targeted at video-subject(s) throughout the entire duration. First, action localization is performed on each frame by identifying patches with homogeneous motion patterns. Thus, a single salient patch is pinpointed per frame. But to avoid viewpoint jitters and flickering, any inter-frame scale or position changes among the patches should be performed gradually over time. This issue is addressed with a polyBézier fitting in 3D space that passes through some chosen pivot timestamps and whose shape is influenced by the in-between control timestamps. To corroborate the effectiveness of the proposed method, we evaluate the video classification task by comparing our dynamic cropping technique with random cropping on three benchmark datasets, viz. UCF-101, HMDB-51 and ActivityNet v1.3. The clip and top-1 accuracy for video classification after our cropping, outperform 3D CNN performances for same-sized random-crop inputs, also surpassing some larger random-crop sizes.

show abstract

Section: Comparison With State-of-the-artsmentioning

confidence: 99%

Unsupervised Action Localization Crop in Video Retargeting for 3D ConvNets

Jana¹,

Bhaumik²,

Mohanta³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, this is generally insufficient for fine-grained tasks. As for example, it is a difficult feat [4] to discriminate similar events like anniversary event and birthday party, both of which showcase some kind of gathering of people and celebration. In such cases, frame-wise object detection may serve to be useful that can identify objects involved throughout the video duration.…”

Section: Introductionmentioning

confidence: 99%

Recent Trends in 2D Object Detection and Applications in Video Event Recognition

Jana¹,

Mohanta²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Object detection serves as a significant step in improving performance of complex downstream computer vision tasks. It has been extensively studied for many years now and current state-of-the-art 2D object detection techniques proffer superlative results even in complex images. In this chapter, we discuss the geometry-based pioneering works in object detection, followed by the recent breakthroughs that employ deep learning. Some of these use a monolithic architecture that takes a RGB image as input and passes it to a feed-forward ConvNet or vision Transformer. These methods, thereby predict class-probability and bounding-box coordinates, all in a single unified pipeline. Two-stage architectures on the other hand, first generate region proposals and then feed it to a CNN to extract features and predict object category and boundingbox. We also elaborate upon the applications of object detection in video event recognition, to achieve better fine-grained video classification performance. Further, we highlight recent datasets for 2D object detection both in images and videos, and present a comparative performance summary of various state-of-the-art object detection techniques.

show abstract