Through Hawks’ Eyes: Synthetically Reconstructing the Visual Field of a Bird in Flight

Abstract

Birds of prey rely on vision to execute flight manoeuvres that are key to their survival, such as intercepting fast-moving targets or navigating through clutter. A better understanding of the role played by vision during these manoeuvres is not only relevant within the field of animal behaviour, but could also have applications for autonomous drones. In this paper, we present a novel method that uses computer vision tools to analyse the role of active vision in bird flight, and demonstrate its use to answer behavioural questions. Combining motion capture data from Harris’ hawks with a hybrid 3D model of the environment, we render RGB images, semantic maps, depth information and optic flow outputs that characterise the visual experience of the bird in flight. In contrast with previous approaches, our method allows us to consider different camera models and alternative gaze strategies for the purposes of hypothesis testing, allows us to consider visual input over the complete visual field of the bird, and is not limited by the technical specifications and performance of a head-mounted camera light enough to attach to a bird’s head in flight. We present pilot data from three sample flights: a pursuit flight, in which a hawk intercepts a moving target, and two obstacle avoidance flights. With this approach, we provide a reproducible method that facilitates the collection of large volumes of data across many individuals, opening up new avenues for data-driven models of animal behaviour.

Perception of motion salience shapes the emergence of collective motions

“…While the MS-based adaptive interaction presents a promising avenue for integrating bionic mechanisms into bio-inspired swarm robotics, we acknowledge that significant efforts are still needed. These efforts include advancing beyond the current position-based measurement of MS to accurately reconstructing the retina-based individual perception 34 , 68 , 69 , as well as bridging the disparity between mathematical modeling of motion salience and perceptual devices on robot 70 , 71 , etc.…”

Fine-scale tracking reveals visual field use for predator detection and escape in collective foraging of pigeon flocks

“…In everyday natural tasks, such as locating food, avoiding predators, and interacting with conspecifics, animals constantly face challenges in deciding when and how to adjust their behaviors to increase their chances of survival (McFarland, 1977). One such behavior is the focusing of attention, or “looking” behavior, which has been relatively understudied in natural tasks due to the technical challenges involved in tracking an animal’s gaze during natural activities (but see Kane et al, 2015; Kano et al, 2018; Miñano et al, 2023; Yorzinski & Platt, 2014). However, this is relatively well studied within the context of vigilance, a scenario where an animal’s survival hinges on effective attentional allocation and where researchers can observe their scanning behavior even in field conditions (Cresswell, 1994; Evans et al, 2018; Inglis & Lazarus, 1981).…”

Brightness cues affect gap negotiation behaviours in zebra finches flying between perches

“…For example, the visual system that guides perching in budgerigars relies specifically on edge detection [ 17 ]. Likewise, Harris’ hawks Parabuteo unicinctus have been reported both to direct their gaze at the edges of obstacles they are avoiding [ 18 ] and to steer towards a clearance of approximately one wing length from an obstacle’s edge [ 19 ]. Nevertheless, in cluttered environments such as forests, there may be many edges to attend to, so it is reasonable to ask whether there are simpler heuristics that might also be used to guide flight.…”