OmniTrack: Real-Time Detection and Tracking of Objects, Text and Logos in Video

Abstract

The automatic detection and tracking of general objects (like persons, animals or cars), text and logos in a video is crucial for many video understanding tasks, and usually real-time processing as required. We propose OmniTrack, an efficient and robust algorithm which is able to automatically detect and track objects, text as well as brand logos in realtime. It combines a powerful deep learning based object detector (YoloV3) with high-quality optical flow methods. Based on the reference YoloV3 C++ implementation, we did some important performance optimizations which will be described. The major steps in the training procedure for the combined detector for text and logo will be presented. We will describe then the OmniTrack algorithm, consisting of the phases preprocessing, feature calculation, prediction, matching and update. Several performance optimizations have been implemented there as well, like doing the object detection and optical flow calculation asynchronously. Experiments show that the proposed algorithm runs in real-time for standard definition (720x576) video on a PC with a Quadro RTX 5000 GPU.

A New Deep Wavefront Based Model for Text Localization in 3D Video

“…Fassold and Germi [31] proposed a method for video text tracking in a real-time environment. Their approach combines deep learning and an object detector to achieve improved results.…”

Detic-Track: Robust Detection and Tracking of Objects in Video

“…The automatic detection and tracking of general objects in a video provides semantic information which is crucial for many high-level computer vision tasks in various application areas like surveillance, autonomous driving, automatic video annotation and brand monitoring. Our proposed Detic-Track algorithm for object detection and tracking is based upon the OmniTrack algorithm [1], which combines the YoloV4 object detector with TV-L1 optical flow and is real-time capable. For the Detic-Track algorithm, we have extended this algorithm in several ways.…”

“…In contrast to YoloV4 (which detects only the 80 MS-COCO classes), Detic is able to detect significantly more object categories. Specifically, we employ a pretrained Detic model which detects the 1, 203 object classes from the LVIS dataset 1 . Furthermore, instead of using the whole bounding box for tracking a detected object, we utilize only the part of the bounding box corresponding to the object mask.…”

The Hyper360 toolset for enriched 360$^\circ$ video

“…Furthermore, even on devices capable of consuming 360 • videos interactively, an user might prefer a lean-back mode, without the need to navigate around actively to explore the content. The initial prototype of the automatic camera path generator (more details can be found in [12]) is based on the information about the scene objects (persons, animals, ...), which is extracted with the method given in [13]. For each scene object, a saliency score is calculated based on several influencing factors (object class and size, motion magnitude, neighbours of object), which indicates the interestingness of the object.…”