Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies

Sadeghian, Amir; Alahi, Alexandre; Savarese, Silvio

doi:10.1109/iccv.2017.41

Cited by 491 publications

(410 citation statements)

References 81 publications

Supporting

Mentioning

406

Contrasting

Order By: Relevance

“…One pronounced example is ROLO [33], which uses YOLOv1 as its feature extractor, combined with LSTMs. Similarly, [34] uses VGG-16 for feature extraction and inputs the 500x1 feature vector into an LSTM. LSTM networks have been shown to provide lower Mean Squared Error in single object and fewer ID switches in multi-object tasks.…”

Section: Related Work a Pedestrian Detection Re-identificationmentioning

confidence: 99%

REVAMP²T: Real-Time Edge Video Analytics for Multicamera Privacy-Aware Pedestrian Tracking

Neff

Mendieta

Mohan

et al. 2020

IEEE Internet Things J.

View full text Add to dashboard Cite

This article presents REVAMP 2 T, Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking, as an integrated end-to-end IoT system for privacybuilt-in decentralized situational awareness. REVAMP 2 T presents novel algorithmic and system constructs to push deep learning and video analytics next to IoT devices (i.e. video cameras). On the algorithm side, REVAMP 2 T proposes a unified integrated computer vision pipeline for detection, re-identification, and tracking across multiple cameras without the need for storing the streaming data. At the same time, it avoids facial recognition, and tracks and re-identifies pedestrians based on their key features at runtime. On the IoT system side, REVAMP 2 T provides infrastructure to maximize hardware utilization on the edge, orchestrates global communications, and provides system-wide re-identification, without the use of personally identifiable information, for a distributed IoT network. For the results and evaluation, this article also proposes a new metric, Accuracy • Efficiency (AE), for holistic evaluation of IoT systems for real-time video analytics based on accuracy, performance, and power efficiency. REVAMP 2 T outperforms current state-of-the-art by as much as thirteen-fold AE improvement.

show abstract

Section: Related Work a Pedestrian Detection Re-identificationmentioning

confidence: 99%

REVAMP²T: Real-Time Edge Video Analytics for Multicamera Privacy-Aware Pedestrian Tracking

Neff

Mendieta

Mohan

et al. 2020

IEEE Internet Things J.

View full text Add to dashboard Cite

show abstract

“…Furthermore, it is worth noting that an association/correspondence strategy based only on object locations is likely to fail to track humans. In this case, more elaborated models considering explicitly the appearance should be taken into account as, for instance, using bi-directional long short-term memories to handle appearance changes [41,42].…”

Section: Object Tracking and Final Augmented Representationmentioning

confidence: 99%

Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework Using Visual and Depth Cues

Martins

Bersan

Campos

et al. 2020

J Intell Robot Syst

View full text Add to dashboard Cite

This paper addresses the problem of building augmented metric representations of scenes with semantic information from RGB-D images. We propose a complete framework to create an enhanced map representation of the environment with object-level information to be used in several applications such as human-robot interaction, assistive robotics, visual navigation, or in manipulation tasks. Our formulation leverages a CNN-based object detector (Yolo) with a 3D model-based segmentation technique to perform instance semantic segmentation, and to localize, identify, and track different classes of objects in the scene. The tracking and positioning of semantic classes is done with a dictionary of Kalman filters in order to combine sensor measurements over time and then providing more accurate maps. The formulation is designed to identify and to disregard dynamic objects in order to obtain a mediumterm invariant map representation. The proposed method was evaluated with collected and publicly available RGB-D data sequences acquired in different indoor scenes. Experimental results show the potential of the technique to produce augmented semantic maps containing several objects (notably doors). We also provide to the community a dataset composed of annotated object classes (doors, fire extinguishers, benches, water fountains) and their positioning, as well as the source code as ROS packages. 1 1 Preprint paper version to appear at Journal of Intelligent & Robotic Systems, available online at: https://doi.

show abstract

“…Our approach to cell tracking is motivated by the now classic work of Jaqaman et al 32 and recent work applying deep learning to object tracking 33 . In these works, object tracking is treated as a linear assignment problem (Figure 2a).…”

Section: Tracking Single Cells With Deep Learning and Linear Programmingmentioning

confidence: 99%

“…Here, we take a supervised deep learning approach to learn an optimal cost function for the linear assignment framework. Our approach was inspired by previous work applying deep learning to object tracking 33 . Building on this work, we make adaptations to deal with the unique features of live-cell imaging data ( Figure 1c).…”

Section: Tracking Single Cells With Deep Learning and Linear Programmingmentioning

confidence: 99%

Caliban: Accurate cell tracking and lineage construction in live-cell imaging experiments with deep learning

Moen

Borba

Miller

et al. 2019

Preprint

View full text Add to dashboard Cite

Live-cell imaging experiments have opened an exciting window into the behavior of living systems. While these experiments can produce rich data, the computational analysis of these datasets is challenging. Single-cell analysis requires that cells be accurately identified in each image and subsequently tracked over time. Increasingly, deep learning is being used to interpret microscopy image with single cell resolution. In this work, we apply deep learning to the problem of tracking single cells in live-cell imaging data. Using crowdsourcing and a human-in-the-loop approach to data annotation, we constructed a dataset of over 11,000 trajectories of cell nuclei that includes lineage information. Using this dataset, we successfully trained a deep learning model to perform cell tracking within a linear programming framework. Benchmarking tests demonstrate that our method achieves state-of-the-art performance on the task of cell tracking with respect to multiple accuracy metrics. Further, we show that our deep learning-based method generalizes to perform cell tracking for both fluorescent and brightfield images of the cell cytoplasm, despite having never been trained on those data types. This enables analysis of live-cell imaging data collected across imaging modalities. A persistent cloud deployment of our cell tracker is available at http://www.deepcell.org.

show abstract

Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies

Cited by 491 publications

References 81 publications

REVAMP²T: Real-Time Edge Video Analytics for Multicamera Privacy-Aware Pedestrian Tracking

REVAMP²T: Real-Time Edge Video Analytics for Multicamera Privacy-Aware Pedestrian Tracking

Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework Using Visual and Depth Cues

Caliban: Accurate cell tracking and lineage construction in live-cell imaging experiments with deep learning

Contact Info

Product

Resources

About

Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies

Cited by 491 publications

References 81 publications

REVAMP2T: Real-Time Edge Video Analytics for Multicamera Privacy-Aware Pedestrian Tracking

REVAMP2T: Real-Time Edge Video Analytics for Multicamera Privacy-Aware Pedestrian Tracking

Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework Using Visual and Depth Cues

Caliban: Accurate cell tracking and lineage construction in live-cell imaging experiments with deep learning

Contact Info

Product

Resources

About

REVAMP²T: Real-Time Edge Video Analytics for Multicamera Privacy-Aware Pedestrian Tracking

REVAMP²T: Real-Time Edge Video Analytics for Multicamera Privacy-Aware Pedestrian Tracking