LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking

Ning, Guanghan; Pei, Jian; Huang, Heng

doi:10.1109/cvprw50498.2020.00525

Cited by 59 publications

(34 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several human pose estimation systems have real-time options: OpenPose ( Cao et al, 2017 ) has a real-time hand/face pose tracker available, and PifPaf ( Kreiss et al, 2019 ) reaches about 10 Hz on COCO (depending on the backbone; Lin et al, 2014 ). On the challenging multi-human PoseTrack benchmark ( Andriluka et al, 2018 ), LightTrack ( Ning et al, 2020 ) reaches less than 1 Hz. However, recent work achieves 3D multi-human pose estimation at remarkable frame rates ( Chen et al, 2020 ), in particular they report an astonishing 154 FPS for 12 cameras with four people in the frame.…”

Section: Discussionmentioning

confidence: 99%

Real-time, low-latency closed-loop feedback using markerless posture tracking

Kane

Lopes²,

Saunders

et al. 2020

eLife

104

View full text Add to dashboard Cite

The ability to control a behavioral task or stimulate neural activity based on animal behavior in real-time is an important tool for experimental neuroscientists. Ideally, such tools are noninvasive, low-latency, and provide interfaces to trigger external hardware based on posture. Recent advances in pose estimation with deep learning allows researchers to train deep neural networks to accurately quantify a wide variety of animal behaviors. Here, we provide a new <monospace>DeepLabCut-Live!</monospace> package that achieves low-latency real-time pose estimation (within 15 ms, >100 FPS), with an additional forward-prediction module that achieves zero-latency feedback, and a dynamic-cropping mode that allows for higher inference speeds. We also provide three options for using this tool with ease: (1) a stand-alone GUI (called <monospace>DLC-Live! GUI</monospace>), and integration into (2) <monospace>Bonsai,</monospace> and (3) <monospace>AutoPilot</monospace>. Lastly, we benchmarked performance on a wide range of systems so that experimentalists can easily decide what hardware is required for their needs.

show abstract

Section: Discussionmentioning

confidence: 99%

Real-time, low-latency closed-loop feedback using markerless posture tracking

Kane

Lopes²,

Saunders

et al. 2020

eLife

104

View full text Add to dashboard Cite

show abstract

“…Similarly, [13] processes two consecutive frames and generates two-frame tracklets that are afterwards combined into multi-frame tracks. A group of MOT methods leverage pose estimation of humans for tracking purposes [15,37,40,50]. However, tracking strategies based on human pose exhibit difficulties as they struggle in correctly estimating pose for low-resolution persons such as the small players that usually appear in real wide-angle soccer videos, as tackled in this work.…”

Section: Related Workmentioning

confidence: 99%

Self-Supervised Small Soccer Player Detection and Tracking

Hurault

Ballester

Haro

2020

Proceedings of the 3rd International Workshop on Multimedia Content Analysis in Sports

View full text Add to dashboard Cite

In a soccer game, the information provided by detecting and tracking brings crucial clues to further analyze and understand some tactical aspects of the game, including individual and team actions. State-of-the-art tracking algorithms achieve impressive results in scenarios on which they have been trained for, but they fail in challenging ones such as soccer games. This is frequently due to the player small relative size and the similar appearance among players of the same team. Although a straightforward solution would be to retrain these models by using a more specific dataset, the lack of such publicly available annotated datasets entails searching for other effective solutions. In this work, we propose a self-supervised pipeline which is able to detect and track low-resolution soccer players under different recording conditions without any need of ground-truth data. Extensive quantitative and qualitative experimental results are presented evaluating its performance. We also present a comparison to several state-of-the-art methods showing that both the proposed detector and the proposed tracker achieve top-tier results, in particular in the presence of small players. Code available at https:// github.com/ samuro95/ Self-Supervised-Small-Soccer-Player-Detection-Tracking.

show abstract

“…is the number of persons whose features are extracted. The skeleton sequence can be obtained by pose estimation algorithms [24][25][26][27] and pose tracking techniques [27][28][29].…”

Section: Action Recognition Using Group-skeleton Datamentioning

confidence: 99%

Group-Skeleton-Based Human Action Recognition in Complex Events

Sun

Chen

2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Human action recognition as an important application of computer vision has been studied for decades. Among various approaches, skeleton-based methods recently attract increasing attention due to their robust and superior performance. However, existing skeleton-based methods ignore the potential action relationships between different persons, while the action of a person is highly likely to be impacted by another person especially in complex events. In this paper, we propose a novel groupskeleton-based human action recognition method in complex events. This method first utilizes multi-scale spatial-temporal graph convolutional networks (MS-G3Ds) to extract skeleton features from multiple persons. In addition to the traditional key point coordinates, we also input the key point speed values to the networks for better performance. Then we use multilayer perceptrons (MLPs) to embed the distance values between the reference person and other persons into the extracted features. Lastly, all the features are fed into another MS-G3D for feature fusion and classification. For avoiding class imbalance problems, the networks are trained with a focal loss. The proposed algorithm is also our solution for the Large-scale Human-centric Video Analysis in Complex Events Challenge. Results on the HiEve dataset show that our method can give superior performance compared to other state-of-the-art methods.

show abstract

LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking

Cited by 59 publications

References 25 publications

Real-time, low-latency closed-loop feedback using markerless posture tracking

Real-time, low-latency closed-loop feedback using markerless posture tracking

Self-Supervised Small Soccer Player Detection and Tracking

Group-Skeleton-Based Human Action Recognition in Complex Events

Contact Info

Product

Resources

About