AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild

Joska, Daniel; Clark, Liam; Muramatsu, Naoya; Jericevich, Ricardo; Nicolls, Fred; Mathis, Alexander; Mathis, Mackenzie Weygandt; Patel, Amir

doi:10.1109/icra48506.2021.9561338

Cited by 36 publications

(22 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…lighting conditions that guarantee species visibility), calibration of—and unlimited power‐supply to—high‐definition automatic recorders such as modern digital cameras. Therefore, laboratory systems are often the initial developmental space for automated technologies, and help pioneering technological advancements that can later be transferred into the field (Joska et al, 2021 ). Small‐scale experimental systems offer a perfect opportunity to test and develop the concept of fully automated workflows (Alisch et al, 2018 ).…”

Section: Combining Technologies To Fully Automate the Monitoring Of M...mentioning

confidence: 99%

Towards the fully automated monitoring of ecological communities

et al. 2022

View full text Add to dashboard Cite

Ecosystems are increasingly exposed to stressors, leading to unprecedented rates of biodiversity decline globally (Capdevila et al., 2022;Díaz et al., 2019). Our ability to reliably forecast ecosystems dynamics is limited by our capacity to understand what governs their composition, dynamics, function and structure (Dietze et al., 2018;Petchey et al., 2015). To drive predictive ecology forward and design appropriate conservation strategies, we therefore need access to long-term, highresolution and standardised information about ecosystems' abiotic and biotic components (Farley et al., 2018;

show abstract

Section: Combining Technologies To Fully Automate the Monitoring Of M...mentioning

confidence: 99%

Towards the fully automated monitoring of ecological communities

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Therefore, we combine datasets, as shown in Figure 2, that cover these two broad interest areas. Specifically, we use: StanfordDogs (21,49), AnimalPose (15), Horse30 (22), AcinoSet (17), Badja (50), AwA (19), Openfield (51), TriMouse (52), FST-LDB-EPM-CSI-OFT (53), BlackMice (54), and in-house data.…”

Section: Pre-training Datasetsmentioning

confidence: 99%

SuperAnimal models pretrained for plug-and-play analysis of animal behavior

Ye¹,

Mathis²,

Mathis³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Animal pose estimation is critical in applications ranging from life science research, agriculture, to veterinary medicine. Compared to human pose estimation, the performance of animal pose estimation is limited by the size of available datasets and the generalization of a model across datasets. Typically different keypoints are labeled regardless of whether the species are the same or not, leaving animal pose datasets to have disjoint or partially overlapping keypoints. As a consequence, a model cannot be used as a plug-and-play solution across datasets. This reality motivates us to develop panoptic animal pose estimation models that are able to predict keypoints defined in all datasets. In this work we propose a simple yet effective way to merge differentially labeled datasets to obtain the largest quadruped and lab mouse pose dataset. Using a gradient masking technique, so called SuperAnimal-models are able to predict keypoints that are distributed across datasets and exhibit strong zero-shot performance. The models can be further improved by (pseudo) labeled fine-tuning. These models outperform ImageNet-initialized models.

show abstract

“…tahs [15], rats [20], and insects [8] have followed. Multi-view Boostrapping [33] has demonstrated how these calibrated multi-camera datasets can be labeled efficiently through a semi-supervised learning paradigm and a small number of hand annotations.…”

Section: Related Workmentioning

confidence: 99%

MBW: Multi-view Bootstrapping in the Wild

Dabhi¹,

Wang²,

Clifford³

et al. 2022

Preprint

View full text Add to dashboard Cite

Labeling articulated objects in unconstrained settings have a wide variety of applications including entertainment, neuroscience, psychology, ethology, and many fields of medicine. Large offline labeled datasets do not exist for all but the most common articulated object categories (e.g., humans). Hand labeling these landmarks within a video sequence is a laborious task. Learned landmark detectors can help, but can be error-prone when trained from only a few examples. Multi-camera systems that train fine-grained detectors have shown significant promise in detecting such errors, allowing for self-supervised solutions that only need a small percentage of the video sequence to be hand-labeled. The approach, however, is based on calibrated cameras and rigid geometry, making it expensive, difficult to manage, and impractical in real-world scenarios. In this paper, we address these bottlenecks by combining a non-rigid 3D neural prior with deep flow to obtain high-fidelity landmark estimates from videos with only two or three uncalibrated, handheld cameras. With just a few annotations (representing 1-2% of the frames), we are able to produce 2D results comparable to state-of-the-art fully supervised methods, along with 3D reconstructions that are impossible with other existing approaches. Our Multi-view Bootstrapping in the Wild (MBW) approach demonstrates impressive results on standard human datasets, as well as tigers, cheetahs, fish, colobus monkeys, chimpanzees, and flamingos from videos captured casually in a zoo. We release the codebase for MBW as well as this challenging zoo dataset consisting image frames of tail-end distribution categories with their corresponding 2D, 3D labels generated from minimal human intervention. * indicates the authors advised equally 36th Conference on Neural Information Processing Systems (NeurIPS 2022) Track on Datasets and Benchmarks.

show abstract

AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild

Cited by 36 publications

References 37 publications

Towards the fully automated monitoring of ecological communities

Towards the fully automated monitoring of ecological communities

SuperAnimal models pretrained for plug-and-play analysis of animal behavior

MBW: Multi-view Bootstrapping in the Wild

Contact Info

Product

Resources

About