T-CNN: Tubelets With Convolutional Neural Networks for Object Detection From Videos

Abstract

The state-of-the-art performance for object detection has been significantly improved over the past two years. Besides the introduction of powerful deep neural networks such as GoogleNet [1] and VGG [2], novel object detection frameworks such as R-CNN [3] and its successors, Fast R-CNN [4] and Faster R-CNN [5], play an essential role in improving the state-of-the-art. Despite their effectiveness on still images, those frameworks are not specifically designed for object detection from videos. Temporal and contextual information of videos are not fully investigated and utilized. In this work, we propose a deep learning framework that incorporates temporal and contextual information from tubelets obtained in videos, which dramatically improves the baseline performance of existing stillimage detection frameworks when they are applied to videos. It is called T-CNN, i.e. tubelets with convolutional neueral networks. The proposed framework won newly introduced object-detectionfrom-video (VID) task with provided data in the ImageNet Large-Scale Visual Recognition Challenge 2015 (ILSVRC 2015). Code is publicly available at https://github.com/myfavouritekk/T-CNN.

Sequence Level Semantics Aggregation for Video Object Detection

“…Several previous works devised various post-processing techniques applied to the results of still image detectors by leveraging temporal information: Kang et al [15,14] proposed to suppress false positive detections via multicontext suppression (MCS) and propagate predicted bounding boxes across frames using the motion calculated by optical flow. Then a temporal convolution neural network is trained to rescore the tubelets generated using visual tracking.…”

“…Another line of work [14] focuses on utilizing optical flow to extract motion information to facilitate object detection. However, such pre-computed optical flow is neither efficient nor task related.…”

Great Ape Detection in Challenging Jungle Camera Trap Footage via Attention-Based Spatial and Temporal Feature Blending

“…Linking single frame detections across the temporal dimension as done by T-CNN [13] constitutes possibly the simplest form of temporal domain exploration. T-CNN essentially runs region-based detectors per frame and enforces motion-based propagation to adjacent frames.…”

Object Detection with Convolutional Neural Networks

“…Therefore, the performance of object detection will affect almost all other computer vision research. A huge amount of effort has been put into its improvements [30,31,32].…”