An autonomous drone for search and rescue in forests using airborne optical sectioning

Abstract

Autonomous drones will play an essential role in human-machine teaming in future search and rescue (SAR) missions. We present a prototype that finds people fully autonomously in densely occluded forests. In the course of 17 field experiments conducted over various forest types and under different flying conditions, our drone found, in total, 38 of 42 hidden persons. For experiments with predefined flight paths, the average precision was 86%, and we found 30 of 34 cases. For adaptive sampling experiments (where potential findings are double-checked on the basis of initial classification confidences), all eight hidden persons were found, leading to an average precision of 100%, whereas classification confidence was increased on average by 15%. Thermal image processing, classification, and dynamic flight path adaptation are computed on-board in real time and while flying. We show that deep learning–based person classification is unaffected by sparse and error-prone sampling within straight flight path segments. This finding allows search missions to be substantially shortened and reduces the image complexity to 1/10th when compared with previous approaches. The goal of our adaptive online sampling technique is to find people as reliably and quickly as possible, which is essential in time-critical applications, such as SAR. Our drone enables SAR operations in remote areas without stable network coverage, because it transmits to the rescue team only classification results that indicate detections and can thus operate with intermittent minimal-bandwidth connections (e.g., by satellite). Once received, these results can be visually enhanced for interpretation on remote mobile devices.

Acceleration-Aware Path Planning with Waypoints

“…This type of problem is well studied in literature and is termed as a minimum time search problem (MTS) [2][3][4][5][6][7][8][9][10][11][12][13][14]. The most prominent objective in these approaches is to optimize the expected time of target detection [3][4][5][6]; however, other alternative approaches involve optimizing the probability of target detection [7][8][9]15], minimizing its counterpart, i.e., probability of nondetection [10,11] or maximizing the information gain [12,13,16]. Various sub-optimal and heuristics-based algorithms such as gradient-based approaches [7,[10][11][12]15], cross-entropy optimization [2,5], Bayesian optimization algorithms [4], ant colony optimization [6], or genetic algorithms [3] have been proposed to address the NP-hard complex problem [13].…”

“…The most prominent objective in these approaches is to optimize the expected time of target detection [3][4][5][6]; however, other alternative approaches involve optimizing the probability of target detection [7][8][9]15], minimizing its counterpart, i.e., probability of nondetection [10,11] or maximizing the information gain [12,13,16]. Various sub-optimal and heuristics-based algorithms such as gradient-based approaches [7,[10][11][12]15], cross-entropy optimization [2,5], Bayesian optimization algorithms [4], ant colony optimization [6], or genetic algorithms [3] have been proposed to address the NP-hard complex problem [13]. These approaches can also be differentiated based on the considered UAV dynamics models, where they either do not consider velocity at all [2,[4][5][6][7][8][9]15], or only consider simple linear velocity models [3,10,11] but not acceleration or deceleration.…”

“…Various sub-optimal and heuristics-based algorithms such as gradient-based approaches [7,[10][11][12]15], cross-entropy optimization [2,5], Bayesian optimization algorithms [4], ant colony optimization [6], or genetic algorithms [3] have been proposed to address the NP-hard complex problem [13]. These approaches can also be differentiated based on the considered UAV dynamics models, where they either do not consider velocity at all [2,[4][5][6][7][8][9]15], or only consider simple linear velocity models [3,10,11] but not acceleration or deceleration.…”

“…Objects of interest (people in search and rescue scenario) at a particular distance often remain occluded by the occluders (such as forests). Airborne optical sectioning (AOS) is a wide synthetic-aperture aerial imaging technique that applies camera drones for the real-time removal of occlusion caused by vegetation, such as forests [15,[17][18][19][20][21][22][23][24][25]. It has been demonstrated as a capable and effective tool in various applications (such as archaeology [17], wildlife observation [21], and search and rescue [15,24]).…”

“…Airborne optical sectioning (AOS) is a wide synthetic-aperture aerial imaging technique that applies camera drones for the real-time removal of occlusion caused by vegetation, such as forests [15,[17][18][19][20][21][22][23][24][25]. It has been demonstrated as a capable and effective tool in various applications (such as archaeology [17], wildlife observation [21], and search and rescue [15,24]). AOS' efficiency concerning the occlusion density, occluder sizes, number of integrated samples, and size of the synthetic aperture has been explained by employing a Drones 2021, 5, 143 2 of 18 randomly distributed statistical model [19,25].…”