Vision-based Model Predictive Control for Unmanned Aerial Vehicles Automatic Trajectory Generation and Tracking

Razzanelli, Matteo; Innocenti, Mario; Pannocchia, Gabriele; Pollini, Lorenzo

doi:10.2514/6.2019-1409

Cited by 3 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The same approach was investigated in [14] to develop a tracking controller for UAVs. The application of the model predictive control (MPC) scheme has been observed in the context of a mobile robot [15] and quadrotor [16]. In the aforementioned scenario, the model predictive control (MPC) was employed to ensure that the visual attribute of the target remains in the intended location within the image.…”

Section: Introductionmentioning

confidence: 99%

Quadrotor UAV Dynamic Visual Servoing Based on Differential Flatness Theory

et al. 2023

View full text Add to dashboard Cite

In this paper, we propose 2D dynamic visual servoing (Dynamic IBVS), where a quadrotor UAV tries to track a moving target using a single facing-down perspective camera. As an application, we propose the tracking of a car-type vehicle. In this case, data related to the altitude and the lateral angles have no importance for the visual system. Indeed, to perform the tracking, we only need to know the longitudinal displacements (along the x and y axes) and the orientation along the z-axis. However, those data are necessary for the quadrotor’s guidance problem. Thanks to the concept of differential flatness, we demonstrate that if we manage to extract the displacements according to the three axes and the orientation according to the yaw angle (the vertical axis) of the quadrotor, we can control all the other variables of the system. For this, we consider a camera equipped with a vertical stabilizer that keeps it in a vertical position during its movement (a gimbaled camera). Other specialized sensors measure information regarding altitude and lateral angles. In the case of classic 2D visual servoing, the elaboration of the kinematic torsor of the quadrotor in no way guarantees the physical realization of instructions, given that the quadrotor is an under-actuated system. Indeed, the setpoint has a dimension equal to six, while the quadrotor is controlled only by four inputs. In addition, the dynamics of a quadrotor are generally very fast, which requires a high-frequency control law. Furthermore, the complexity of the image processing stage can cause delays in motion control, which can lead to target loss. A new dynamic 2D visual servoing method (Dynamic IBVS) is proposed. This method makes it possible to generate in real time the necessary movements for the quadrotor in order to carry out the tracking of the target (vehicle) using a single point of this target as visual information. This point can represent the center of gravity of the target or any other part of it. A control by flatness has been proposed, which guarantees the controllability of the system and ensures the asymptotic convergence of the generated trajectory in the image plane. Numerical simulations are presented to show the effectiveness of the proposed control strategy.

show abstract

Section: Introductionmentioning

confidence: 99%

Quadrotor UAV Dynamic Visual Servoing Based on Differential Flatness Theory

et al. 2023

View full text Add to dashboard Cite

show abstract

“…A nonlinear model predictive controller was applied to an underwater vehicle to generate a desired velocity while satisfying the constraints of the visibility [ 20 ]. Similarly, MPC has been used to control a mobile robot while tracking a stationary feature point [ 21 ], to keep a visual feature of the target in the curtain position of the image [ 22 , 23 ], and to maximize the visibility of a target and minimize the velocity of its image feature when utilizing quadrotors. Artificial patterns have been used to predict all of the missing feature points due to occlusion problems and to ensure the execution of IBVS for navigation [ 24 ].…”

Section: Introductionmentioning

confidence: 99%

Visual Servoing of a Moving Target by an Unmanned Aerial Vehicle

Chen

Hung

Yang

et al. 2021

Sensors

View full text Add to dashboard Cite

To track moving targets undergoing unknown translational and rotational motions, a tracking controller is developed for unmanned aerial vehicles (UAVs). The main challenges are to control both the relative position and orientation between the target and the UAV to within desired values, and to guarantee that the generated control input to the UAV is feasible (i.e., below its motion capability). Moreover, the UAV is controlled to ensure that the target always remains within the field of view of the onboard camera. These control objectives were achieved by developing a nonlinear-model predictive controller, in which the future motion of the target is predicted by quadratic programming (QP). Since constraints of the feature vector and the control input are considered when solving the optimal control problem, the control inputs can be bounded and the target can remain inside the image. Three simulations were performed to compare the efficacy and performance of the developed controller with a traditional image-based visual servoing controller.

show abstract

“…Furthermore, given the known change in distance between image frames, it is also possible to estimate the relative velocity and acceleration of the target. With this, a control system can be developed to enable an RPA to perform desirable maneuvers based on the position of the target, such as maintaining a relative position or orbiting around it, as presented by Razzanelli et al (Razzanelli, Innocenti, Pannocchia, & Pollini, 2019). In this work, a model predictive controller leveraging the information provided by a vision system was developed to provide trajectory generation for following and tracking another aircraft.…”

Section: Target Detection and Trackingmentioning

confidence: 99%

Development of a Computer Vision Framework for Improved Remotely Piloted Aircraft Operations

Ooi¹

View full text Add to dashboard Cite

This thesis proposes a computer vision framework to enable improved operations of Remotely Piloted Aircraft equipped with onboard image sensors. The main use of payload image sensors is to provide visual imagery data to the system for real-time or postprocessing applications; the application of an image quality metric and the ground sampling distance of the image sensor can be used to predict the performance of an image sensor in enabling the image classification task. This information is used to determine the mission-specific operational envelope of the aircraft, to ensure that visual data quality requirements are met. The application of a convolutional neural network for image processing is also presented. Finally, a vision-based positioning system is developed; it achieves an average position estimation difference of 18 cm compared to a commercially available indoor localization system and provides a position update rate at 12 Hz. viii 4.3

show abstract

Vision-based Model Predictive Control for Unmanned Aerial Vehicles Automatic Trajectory Generation and Tracking

Cited by 3 publications

References 24 publications

Quadrotor UAV Dynamic Visual Servoing Based on Differential Flatness Theory

Quadrotor UAV Dynamic Visual Servoing Based on Differential Flatness Theory

Visual Servoing of a Moving Target by an Unmanned Aerial Vehicle

Development of a Computer Vision Framework for Improved Remotely Piloted Aircraft Operations

Contact Info

Product

Resources

About