Bio-inspired vision based robot control using featureless estimations of time-to-contact

Abstract

Marvelous vision based dynamic behaviors of insects and birds such as perching, landing, and obstacle avoidance have inspired scientists to propose the idea of time-to-contact, which is defined as the time for a moving observer to contact an object or surface if the current velocity is maintained. Since with only a vision sensor, time-to-contact can be directly estimated from consecutive images, it is widely used for a variety of robots to fulfill various tasks such as obstacle avoidance, docking, chasing, perching and landing. However, most of existing methods to estimate the time-to-contact need to extract and track features during the control process, which is time-consuming and cannot be applied to robots with limited computation power. In this paper, we adopt a featureless estimation method, extend this method to more general settings with angular velocities, and improve the estimation results using Kalman filtering. Further, we design an error based controller with gain scheduling strategy to control the motion of mobile robots. Experiments for both estimation and control are conducted using a customized mobile robot platform with low-cost embedded systems. Onboard experimental results demonstrate the effectiveness of the proposed approach, with the robot being controlled to successfully dock in front of a vertical wall. The estimation and control methods presented in this paper can be applied to computation-constrained miniature robots for agile locomotion such as landing, docking, or navigation.

A direct optic flow-based strategy for inverse flight altitude estimation with monocular vision and IMU measurements

“…Instead of relying on feature detection and tracking method, it has been shown in [30,31] that we can directly combine the optic flow constraint in equation (1) with the equation describing the camera motion (in the scenario described in figure 1, this corresponds to equation (5)). If we let I ij denote the pixel intensity at I u i , v j , t , the result directly relates the image gradients to the visual observables and the camera's attitude: (7).…”

“…Furthermore, in the case that the camera axis is almost vertical, such as a downward-looking camera attached to a hovering quadrotor, we have R 31 , R 32 R 33 . Keeping only the first order terms (see [33] for a complete derivation), equation (4), together with equation (3), simplifies to Figure 1.…”

“…For simplicity, we treated them as constant and they were experimentally tuned. The gyroscopic readings were used for the angular velocities, the accelerometer outputs were taken for a x , a y , and a z needed for the matrix F in equation (23), and the pitch and roll angles from the IMU fusion used for flight attitude stabilization were also used as R 31 and R 32 for the matrix F. Moreover, a 3 rd -order Butterworth low pass filter with a cutoff frequency of 6 Hz was applied to all IMU measurements and computed visual observables in order to minimize the measurement noise and vibration effects. To investigate the performance of the proposed estimation schemes, we carried out the flight tests with three flying patterns: hovering, vertical, and horizontal flights.…”

“…Unlike applications on mobile robots or controlled experiments [24,31], in the context of flying robots, one major challenge in achieving accurate estimates of the inverse distance from the combination of IMU measurements and the images is owing to the present vibrations in flight. In evidence of this, in [21], it was reported that the RMS errors in velocity estimates in flight were higher than those from a handheld experiment, citing the vibrations and motion blur as the cause of inaccuracies.…”

“…Alternative to the LK method, there exists an alternative optic flow method developed by Horn et al [30], which directly provides time-to-contact information from image gradients without the need to identify and track image features. This principle has been extended to estimate the time-to-contact (without resolving for a scale factor) for motion control of mobile robots in [31].…”