3D pose estimation of bats in the wild

Abstract

Vision-based methods have gained popularity as a tool for helping to analyze the behavior of bats. Though, for bats in the wild, there are still no tools capable of estimating and subsequently analyzing articulated 3D bat pose. We propose a model-based multi-view articulated 3D bat pose estimation framework for this novel problem. Key challenges include the large search space associated with articulated 3D pose, the ambiguities that arise from 2D projections of 3D bodies, and the low resolution image data we have available. Our method uses multi-view camera geometry and temporal constraints to reduce the state space of possible articulated 3D bat poses and finds an optimal set using a Markov Random Field based model.Our experiments use real video data of flying bats and gold-standard annotations by a bat biologist. Our results show, for the first time in the literature, articulated 3D pose estimates being generated automatically for video sequences of bats flying in the wild. The average differences in body orientation and wing joint angles, between estimates produced by our method and those based on gold-standard annotations, ranged from 16• -21 • (i.e., ≈ 17% -23%) for orientation and 14 • -26 • (i.e., ≈ 7% -14%) for wing joint angles.

Vision System for Coarsely Estimating Motion Parameters for Unknown Fast Moving Objects in Space

“…One important factor in MOT is the assumed target. According to the specific application, the target can be pedestrians on the street [ 2 , 3 ], vehicles on the road [ 4 ], bats in the sky [ 5 , 6 ], fish under water [ 7 ], etc. Methods in these works are typically based on the prior knowledge of the specific class of assumed targets.…”

“…Methods in these works are typically based on the prior knowledge of the specific class of assumed targets. This knowledge can be either explicitly specified by human engineering [ 3 , 4 , 7 ], or obtained though learning process [ 6 ]. The limitation is within the capability of generalization, i.e., target specific knowledge can not be easily adapted to different targets.…”

Discovering useful parts for pose estimation in sparsely annotated datasets

“…More automated approaches for pose estimation of flying animals include [6,15,26,27,29]. In Ristroph et al [29] multiple views of a fruit fly are segmented and then back projected for visual hull reconstruction.…”

“…Fontaine et al [15] track the 3D pose of a fly over time by registering a 3D articulated graphics model with segmented image data. A similar approach was used by us for 3D pose estimation of bats [6].…”

“…For example, researchers study the group behavior and obstacle avoidance abilities of bats [6,7,24,33], the maneuverability of cliff swallows involved in a chase [30], and the flight performance of hawkmoths under varying wind tunnel conditions [26,27]. Enabling this research to take place are camera systems, which have been essential for the observation of flying animals in both lab conditions and natural habitats [6,7,26,27,30,33]. Analyses of the datasets captured by these camera systems have increasingly been assisted by computer vision algorithms, allowing researchers to save time and labor on tasks that algorithms can do automatically with sufficient accuracy.…”