Static and Dynamic Algorithms for Terrain Classification in UAV Aerial Imagery

Abstract

The ability to precisely classify different types of terrain is extremely important for Unmanned Aerial Vehicles (UAVs). There are multiple situations in which terrain classification is fundamental for achieving a UAV's mission success, such as emergency landing, aerial mapping, decision making, and cooperation between UAVs in autonomous navigation. Previous research works describe different terrain classification approaches mainly using static features from RGB images taken onboard UAVs. In these works, the terrain is classified from each image taken as a whole, not divided into blocks; this approach has an obvious drawback when applied to images with multiple terrain types. This paper proposes a robust computer vision system to classify terrain types using three main algorithms, which extract features from UAV's downwash effect: Static textures-Gray-Level Co-Occurrence Matrix (GLCM), Gray-Level Run Length Matrix (GLRLM) and Dynamic textures-Optical Flow method. This system has been fully implemented using the OpenCV library, and the GLCM algorithm has also been partially specified in a Hardware Description Language (VHDL) and implemented in a Field Programmable Gate Array (FPGA)-based platform. In addition to these feature extraction algorithms, a neural network was designed with the aim of classifying the terrain into one of four classes. Lastly, in order to store and access all the classified terrain information, a dynamic map, with this information was generated. The system was validated using videos acquired onboard a UAV with an RGB camera.Several methods for terrain classification have been proposed in different works. Texture algorithms, such as those proposed in [2][3][4][5], have been widely recommended to emphasize the high and low frequencies of the images, supporting image classification. Other algorithms use color information to classify terrains, such as presented in [6], which is able to distinguish four different terrain types within an image. During this process, each channel's pixel is divided by the square root of its own three channels intensity. The final result will emphasize the color that most represents the terrain type (eg, blue for water). Additionally, frequency domain [7,8], segmentation [6,9,10], bayesian network [11], and Hyperspectal Images [12] can also be used in terrain classification.Other types of sensors such as LiDAR [13][14][15][16] can complement the classification decision. Algorithms that use laser scanners proved to be qualified to accurately distinguish between water and non-water terrains [13][14][15][16]. However, shallow water terrains increase the decision error due to laser reflection, which leads to a misclassification as non-water terrain.Although prior research work has proposed many good solutions for terrain classification, there is still a gap regarding the study of dynamic terrain. The previously mentioned algorithms suffer from a high sensitivity to changes in the environment, mainly due to changes in brightness, color and texture. Recent works [17,18] ...

A Survey on UAV Computing Platforms: A Hardware Reliability Perspective

“…Terrain classification is important for an emergency landing, aerial mapping, decision making, and cooperation between UAVs in autonomous navigation systems. Using three algorithms (Gray-Level Co-Occurrence Matrix, Gray-Level Run Length Matrix, and Flow), the research [36] provided a complete solution for terrain classification in differentiating among the four terrain types (water, vegetation, asphalt, and sand). Their proposed solution developed on the FPGA achieved a 95.14% success rate in train classification using the OpenCV library.…”

Fabric Defect Detection With Deep Learning and False Negative Reduction

“…The latest technological developments in processing power and volume of data facilitate the adoption of learning approaches. These methods try to find patterns in the extracted characteristics of the fabric image and therefore can be used standalone or as a complement to other methods like LBP [26] and GLCM [27]. SVMs [28], Feed-forward Networks (FFNs) [29], [30] and CNNs [31]- [33] are some of the methods included in this type of approaches.…”

Adversarial Patch Attack on Multi-Scale Object Detection for Remote Sensing Image

“…The continuous development of aerial photography technology makes it possible for people to collect numerous high-resolution remote sensing images, which contributes a lot to many important applications in the remote sensing field [1][2][3]. Some typical applications include classification [4,5], image segmentation [6] and object detection [7][8][9][10]. Specifically, object detection, which tries to precisely estimate the class and locations of objects contained in each image, is one primary task [11][12][13].…”