Search and rescue with airborne optical sectioning

Schedl, David C.; Kurmi, Indrajit; Bimber, Oliver

doi:10.1038/s42256-020-00261-3

Cited by 22 publications

(71 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Short battery life restricted flying times to a maximum of 15 to 20 min. Professional drones with (34), resampled to 1D SA segments, for various path lengths N. GT is the number of ground truth labels, TP are the true positives, FP are the false positives, and AP is the AP score. F9 is a flight over an empty (no hidden persons) forest.…”

Section: Discussionmentioning

confidence: 99%

“…In (34), we sampled 2D SA areas of 30 m by 30 m (this equals the ground coverage of our thermal camera's field of view at an altitude of about 35 m above ground level) with a sampling density of 1 m by 3 m. The high number of samples (about 300 images) and precise computer vision-based pose estimation ensured effective occlusion removal and a focused appearance of persons in the corresponding integral images (Fig. 2, A and B).…”

Section: D Sa Classificationmentioning

confidence: 99%

“…In the second series of experiments, we validated our findings in practice in the course of eight new test flights (covering a total area of about 6 ha) over various forest types, where all processing was carried out in real time on the drone and during flight but where the flight paths were still predefined and static (following fixed waypoints). Here, we used test flights from (34) for training (F0, F2 to F6, F8, F10, and F11) and training validation (F1 and F9) and, again, augmented with path length variations from N = 1 to 30. During each new test flight, the drone sampled segments of 30-m-long 1D SAs with 1 sample/m along defined waypoints.…”

Section: Predefined Searchmentioning

confidence: 99%

“…The overall AP score was lower than in the first experiments (86% versus 93%). This is due to Table 1 comparing test flights of our previous offline classification study presented in (34), where compass errors were corrected manually after recording. The data of the new test flights in Table 2 were computed fully automatically during flights and contain compass 2 increases to 92% (shown in parentheses).…”

Section: Predefined Searchmentioning

confidence: 99%

“…The main software running on the SoCC was implemented using Python, whereas submodules, such as the drone communication protocol and integral computation, were implemented using C, OpenGL, and C++ and integrated using Cython. How integral images are computed was explained in detail in (28,30,34). On the SoCC, integrals are computed by projecting and averaging all single images on the ground surface (i.e., DEM or focal plane) using OpenGL's projective texture mapping technique (68).…”

Section: Idmentioning

confidence: 99%

See 4 more Smart Citations

An autonomous drone for search and rescue in forests using airborne optical sectioning

2021

Self Cite

View full text Add to dashboard Cite

Autonomous drones will play an essential role in human-machine teaming in future search and rescue (SAR) missions. We present a prototype that finds people fully autonomously in densely occluded forests. In the course of 17 field experiments conducted over various forest types and under different flying conditions, our drone found, in total, 38 of 42 hidden persons. For experiments with predefined flight paths, the average precision was 86%, and we found 30 of 34 cases. For adaptive sampling experiments (where potential findings are double-checked on the basis of initial classification confidences), all eight hidden persons were found, leading to an average precision of 100%, whereas classification confidence was increased on average by 15%. Thermal image processing, classification, and dynamic flight path adaptation are computed on-board in real time and while flying. We show that deep learning–based person classification is unaffected by sparse and error-prone sampling within straight flight path segments. This finding allows search missions to be substantially shortened and reduces the image complexity to 1/10th when compared with previous approaches. The goal of our adaptive online sampling technique is to find people as reliably and quickly as possible, which is essential in time-critical applications, such as SAR. Our drone enables SAR operations in remote areas without stable network coverage, because it transmits to the rescue team only classification results that indicate detections and can thus operate with intermittent minimal-bandwidth connections (e.g., by satellite). Once received, these results can be visually enhanced for interpretation on remote mobile devices.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: D Sa Classificationmentioning

confidence: 99%

Section: Predefined Searchmentioning

confidence: 99%

Section: Predefined Searchmentioning

confidence: 99%

Section: Idmentioning

confidence: 99%

See 3 more Smart Citations

An autonomous drone for search and rescue in forests using airborne optical sectioning

2021

Self Cite

View full text Add to dashboard Cite

show abstract

Deep learning with RGB and thermal images onboard a drone for monitoring operations

Speth

Gonçalves

Rigault

et al. 2022

Journal of Field Robotics

View full text Add to dashboard Cite

This article describes the artificial intelligence (AI) component of a drone for monitoring and patrolling tasks associated with disaster relief missions in specific restricted disaster scenarios, as specified by the Advanced Robotics Foundation in Japan. The AI component uses deep learning models for environment recognition and object detection. For environment recognition, we use semantic segmentation, or pixel‐wise labeling, based on RGB images. Object detection is key for detecting and locating people in need. Since people are relatively small objects from the drone perspective, we use both RGB and thermal images. To train our models, we created a novel multispectral and publicly available data set of people. We used a geo‐location method to locate people on the ground. The semantic segmentation models were extensively tested using different feature extractors. We created two dedicated data sets, which we have made publicly available. Compared with the baseline model, the best‐performing model could increase the mean intersection over union (IoU) by 1.3%. Furthermore, we compared two types of person detection models. The first one is an ensemble model that combines RGB and thermal information via “late fusion”; the second one is a 4‐channel model that combines these two types of information in an “early fusion” manner. The results suggest that the 4‐channel model had a 40.6% increase of average precision for stricter IoU values (0.75) compared with the ensemble model and a 5.8% increase in the average precision compared with the thermal model. All models were deployed and tested on the NVIDIA AGX Xavier platform. To the best of our knowledge, this study was the first to use both RGB and thermal data from the perspective of a drone for monitoring tasks.

show abstract

An Occlusion Signal-Processing Framework Based on UAV Sampling for Improving Rendering Quality of Views

Liu,

Yan,

Wang

et al. 2024

Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

View full text Add to dashboard Cite

Search and rescue with airborne optical sectioning

Cited by 22 publications

References 52 publications

An autonomous drone for search and rescue in forests using airborne optical sectioning

An autonomous drone for search and rescue in forests using airborne optical sectioning

Deep learning with RGB and thermal images onboard a drone for monitoring operations

An Occlusion Signal-Processing Framework Based on UAV Sampling for Improving Rendering Quality of Views

Contact Info

Product

Resources

About