Unsupervised Human Detection with an Embedded Vision System on a Fully Autonomous UAV for Search and Rescue Operations

Lygouras, Eleftherios; Santavas, Nicholas; Taitzoglou, Anastasios; Tarchanidis, Konstantinos; Mitropoulos, Athanasios C.; Γαστεράτος, Αντώνιος

doi:10.3390/s19163542

Cited by 185 publications

(90 citation statements)

References 64 publications

(75 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Sulistijono et al [17] presented a study to detect disaster victims in highly cluttered scenarios. Lygouras et al [18] also addressed a similar problem of detecting swimmers in disaster areas in the presence of complex backgrounds. Both studies proposed to use deep learning techniques for victim detection and drones for efficient data capture.…”

Section: Related Workmentioning

confidence: 99%

Detection and Localisation of Life Signs from the Air Using Image Registration and Spatio-Temporal Filtering

et al. 2020

View full text Add to dashboard Cite

In search and rescue operations, it is crucial to rapidly identify those people who are alive from those who are not. If this information is known, emergency teams can prioritize their operations to save more lives. However, in some natural disasters the people may be lying on the ground covered with dust, debris, or ashes making them difficult to detect by video analysis that is tuned to human shapes. We present a novel method to estimate the locations of people from aerial video using image and signal processing designed to detect breathing movements. We have shown that this method can successfully detect clearly visible people and people who are fully occluded by debris. First, the aerial videos were stabilized using the key points of adjacent image frames. Next, the stabilized video was decomposed into tile videos and the temporal frequency bands of interest were motion magnified while the other frequencies were suppressed. Image differencing and temporal filtering were performed on each tile video to detect potential breathing signals. Finally, the detected frequencies were remapped to the image frame creating a life signs map that indicates possible human locations. The proposed method was validated with both aerial and ground recorded videos in a controlled environment. Based on the dataset, the results showed good reliability for aerial videos and no errors for ground recorded videos where the average precision measures for aerial videos and ground recorded videos were 0.913 and 1 respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Detection and Localisation of Life Signs from the Air Using Image Registration and Spatio-Temporal Filtering

et al. 2020

View full text Add to dashboard Cite

show abstract

“…After detection of corresponding regions, they analyze the visible spectrum using a cascade of boosted classifiers working with Haar-like features. In [16] the authors proposed a real-time model for the detection swimmers in open water using unsupervised learning. Detecting humans from aerial images in [17] was managed by pyramidal feature extraction of SSD for human detection and action recognition, while in [18] the authors proposed a model for human detection based on color and depth data.…”

Section: Related Workmentioning

confidence: 99%

Visual-Based Person Detection for Search-and-Rescue with UAS: Humans vs. Machine Learning Algorithm

et al. 2020

View full text Add to dashboard Cite

Unmanned Aircraft Systems (UASs) have been recognized as an important resource in search-and-rescue (SAR) missions and, as such, have been used by the Croatian Mountain Search and Rescue (CMRS) service for over seven years. The UAS scans and photographs the terrain. The high-resolution images are afterwards analyzed by SAR members to detect missing persons or to find some usable trace. It is a drawn out, tiresome process prone to human error. To facilitate and speed up mission image processing and increase detection accuracy, we have developed several image-processing algorithms. The latest are convolutional neural network (CNN)-based. CNNs were trained on a specially developed image database, named HERIDAL. Although these algorithms achieve excellent recall, the efficiency of the algorithm in actual SAR missions and its comparison with expert detection must be investigated. A series of mission simulations are planned and recorded for this purpose. They are processed and labelled by a developed algorithm. A web application was developed by which experts analyzed raw and processed mission images. The algorithm achieved better recall compared to an expert, but the experts achieved better accuracy when they analyzed images that were already processed and labelled.

show abstract

“…The main purpose of this system was to identify the work target and update the relative position of the work target. We needed to use different recognition methods in different scenarios because of the difference of target recognition, so in this design, the visual system was not the main content, and the visual recognition methods of different job targets were introduced in several documents [42,43]. Therefore, a simple method was introduced to verify the system.…”

Section: Visual Systemmentioning

confidence: 99%

Obtaining World Coordinate Information of UAV in GNSS Denied Environments

Chen

Tian

Liang

et al. 2020

Sensors

View full text Add to dashboard Cite

GNSS information is vulnerable to external interference and causes failure when unmanned aerial vehicles (UAVs) are in a fully autonomous flight in complex environments such as high-rise parks and dense forests. This paper presents a pan-tilt-based visual servoing (PBVS) method for obtaining world coordinate information. The system is equipped with an inertial measurement unit (IMU), an air pressure sensor, a magnetometer, and a pan-tilt-zoom (PTZ) camera. In this paper, we explain the physical model and the application method of the PBVS system, which can be briefly summarized as follows. We track the operation target with a UAV carrying a camera and output the information about the UAV’s position and the angle between the PTZ and the anchor point. In this way, we can obtain the current absolute position information of the UAV with its absolute altitude collected by the height sensing unit and absolute geographic coordinate information and altitude information of the tracked target. We set up an actual UAV experimental environment. To meet the calculation requirements, some sensor data will be sent to the cloud through the network. Through the field tests, it can be concluded that the systematic deviation of the overall solution is less than the error of GNSS sensor equipment, and it can provide navigation coordinate information for the UAV in complex environments. Compared with traditional visual navigation systems, our scheme has the advantage of obtaining absolute, continuous, accurate, and efficient navigation information at a short distance (within 15 m from the target). This system can be used in scenarios that require autonomous cruise, such as self-powered inspections of UAVs, patrols in parks, etc.

show abstract

Unsupervised Human Detection with an Embedded Vision System on a Fully Autonomous UAV for Search and Rescue Operations

Cited by 185 publications

References 64 publications

Detection and Localisation of Life Signs from the Air Using Image Registration and Spatio-Temporal Filtering

Detection and Localisation of Life Signs from the Air Using Image Registration and Spatio-Temporal Filtering

Visual-Based Person Detection for Search-and-Rescue with UAS: Humans vs. Machine Learning Algorithm

Obtaining World Coordinate Information of UAV in GNSS Denied Environments

Contact Info

Product

Resources

About