Wireless steerable vision for live insects and insect-scale robots

Abstract

Vision serves as an essential sensory input for insects but consumes substantial energy resources. The cost to support sensitive photoreceptors has led many insects to develop high visual acuity in only small retinal regions and evolve to move their visual systems independent of their bodies through head motion. By understanding the trade-offs made by insect vision systems in nature, we can design better vision systems for insect-scale robotics in a way that balances energy, computation, and mass. Here, we report a fully wireless, power-autonomous, mechanically steerable vision system that imitates head motion in a form factor small enough to mount on the back of a live beetle or a similarly sized terrestrial robot. Our electronics and actuator weigh 248 milligrams and can steer the camera over 60° based on commands from a smartphone. The camera streams “first person” 160 pixels–by–120 pixels monochrome video at 1 to 5 frames per second (fps) to a Bluetooth radio from up to 120 meters away. We mounted this vision system on two species of freely walking live beetles, demonstrating that triggering image capture using an onboard accelerometer achieves operational times of up to 6 hours with a 10–milliamp hour battery. We also built a small, terrestrial robot (1.6 centimeters by 2 centimeters) that can move at up to 3.5 centimeters per second, support vision, and operate for 63 to 260 minutes. Our results demonstrate that steerable vision can enable object tracking and wide-angle views for 26 to 84 times lower energy than moving the whole robot.

Mechanisms of punctuated vision in fly flight

“…However, less emphasis has been placed on one of the most defining features of insect vision: insect eyes move via head movements, and are thus active visual sensors (Bajcsy, 1988). As current micro-air vehicles have recently begun to incorporate movable sensors, roboticists and engineers should consider saccade-inspired hybrid control for vision systems with saturation nonlinearities (Iyer et al, 2020). For example, a recent study developed a wireless steerable vision sensor for insect-scale flying systems (Iyer et al, 2020).…”

“…As current micro-air vehicles have recently begun to incorporate movable sensors, roboticists and engineers should consider saccade-inspired hybrid control for vision systems with saturation nonlinearities (Iyer et al, 2020). For example, a recent study developed a wireless steerable vision sensor for insect-scale flying systems (Iyer et al, 2020). This sensor can be actuated through ~60° in yaw via a piezo-based actuator, and thus has saturation constraints analogous to the mechanics of an insect head.…”

A Nearly Packaging‐Free Design Paradigm for Light, Powerful, and Energy‐Dense Primary Microbatteries

“…Commercial micropouch cells have been used to realize several insect‐sized microrobotic platforms that crawl, however, the lightest of these batteries were ≈5× heavier than our microbatteries but still had lower total energy output because of their ≈7× lower energy density. [ 3,36–38 ] By replacing these cells with our microbatteries of the same mass, the operation time of the microrobot would be extended from 1.4 to ≈7 min. We demonstrate that our microbatteries provide minimal disruption to insect‐sized crawlers by attaching a microbattery to a young European mantis ( Mantis religiosa ).…”

Estimating the effect of tracking tag weight on insect movement using video analysis: A case study with a flightless orthopteran

“…The use of UAVs and other ongoing technological advances are creating further possibilities for tracking the movement of insects and will likely increase the use of tracking tags in insect ecology [11,20,[71][72][73]. Furthermore, there are non-tracking cases in which insects are used to carry devices [20,21]. We agree with other researchers [3,19,37] that the effects of tags are likely to be species-specific, i.e., there are no general thresholds or rules for tag/body mass ratios that can be applied to all species.…”