Combining temporal planning with probabilistic reasoning for autonomous surveillance missions

Abstract

It is particularly challenging to devise techniques for underpinning the behaviour of autonomous vehicles in surveillance missions as these vehicles operate in uncertain and unpredictable environments where they must cope with little stability and tight deadlines in spite of their restricted resources. State-of-the-art techniques typically use probabilistic algorithms that suffer a high computational cost in complex real-world scenarios. To overcome these limitations, we propose a hybrid approach that combines the probabilistic reasoning based on the target motion model offered by Monte Carlo simulation with long-term strategic capabilities provided by automated task planning. We demonstrate our approach by focusing on one particular surveillance mission, search-and-tracking, and by using two different vehicles, a fixed-wing UAV deployed in simulation and the "Parrot AR.Drone2.0" quadcopter deployed in a physical environment. Our experimental results show that our unique way of integrating probabilistic and deterministic reasoning pays off when we tackle realistic missions. Electronic supplementary materialThe online version of this article

Policy-Gradient and Actor-Critic Based State Representation Learning for Safe Driving of Autonomous Vehicles

“…All past and present states are in the continuous state-space [ 34 ]. The parameter values, that is, the set of velocities in a given timestep, at present and past instances for a given state S are obtained for the trajectory followed by the vehicle described by [ 20 , 40 ]: where is the difference between two subsequent timeframes while the vehicle navigates the trajectory. These parameters are used to calculate the optimal value function and optimal Q-value .…”

“…The episode ends as soon as the vehicle steers off the road. The episode termination also prevents the vehicular agent from exploring regions that do not contribute at all to effectively learn the driving task [ 40 ]. If a VAE feature extractor is trained after each episode, the distribution of features is not stationary.…”

“…In DDPG, this baseline is a stationary value that does not update with experience. In SAC, a baseline is estimated from experience, making the method an actor-critic method where the vehicle updates its parameters after each step taken in the driving environment [ 40 ]. The Bellman equations allow comparing the results of taking a different action in each state and assist in negating the wrong step, causing the agent to strengthen how it selects the apparent best action.…”

Optimization of Pesticide Spraying Tasks via Multi-UAVs Using Genetic Algorithm

“…It is also the most convenient control strategy to implement and is widely applied to tasks for covered areas [55]. There are two implementations of this strategy, namely, parallel track search and creeping line search [56].…”

Intermittent Information-Driven Multi-Agent Area-Restricted Search

“…The earliest theoretical studies of search strategies [ 9 , 10 ] were based on systematic search along a predetermined (deterministic) path, such as the parallel sweep or the Archimedean spiral [ 3 , 11 ]. The search patterns of animals, on the contrary, are random rather than deterministic.…”