Solar Panel Detection within Complex Backgrounds Using Thermal Images Acquired by UAVs

Abstract

The installation of solar plants everywhere in the world increases year by year. Automated diagnostic methods are needed to inspect the solar plants and to identify anomalies within these photovoltaic panels. The inspection is usually carried out by unmanned aerial vehicles (UAVs) using thermal imaging sensors. The first step in the whole process is to detect the solar panels in those images. However, standard image processing techniques fail in case of low-contrast images or images with complex backgrounds. Moreover, the shades of power lines or structures similar to solar panels impede the automated detection process. In this research, two self-developed methods are compared for the detection of panels in this context, one based on classical techniques and another one based on deep learning, both with a common post-processing step. The first method is based on edge detection and classification, in contrast to the second method is based on training a region based convolutional neural networks to identify a panel. The first method corrects for the low contrast of the thermal image using several preprocessing techniques. Subsequently, edge detection, segmentation and segment classification are applied. The latter is done using a support vector machine trained with an optimized texture descriptor vector. The second method is based on deep learning trained with images that have been subjected to three different pre-processing operations. The postprocessing use the detected panels to infer the location of panels that were not detected. This step selects contours from detected panels based on the panel area and the angle of rotation. Then new panels are determined by the extrapolation of these contours. The panels in 100 random images taken from eleven UAV flights over three solar plants are labeled and used to evaluate the detection methods. The metrics for the new method based on classical techniques reaches a precision of 0.997, a recall of 0.970 and a F1 score of 0.983. The metrics for the method of deep learning reaches a precision of 0.996, a recall of 0.981 and a F1 score of 0.989. The two panel detection methods are highly effective in the presence of complex backgrounds.

Solar Panels String Predictive and Parametric Fault Diagnosis Using Low-Cost Sensors

“…In this sense, methods based on infrared (IR) and electroluminescence (EL) thermography have been used effectively. However, these techniques are not without problems as they require good technical training for the maintenance teams involved [ 13 ]. They also present notable differences in the quality of the analyses depending on whether they are carried out in the outdoor or indoor environment and also restrictions to immediate detection when used manually and causing interruptions in normal operation [ 14 ].…”

Computer vision tool for detection, mapping, and fault classification of photovoltaics modules in aerial IR videos

“…The most popular method is binary thresholding of image intensities to obtain segmentation masks of the PV modules [13][14][15]19,24,25 . Vega Díaz et al 21 detect rectangular candidate contours by thresholding, extract texture features and classify them with a Support Vector Machine (SVM). Other works find edges of PV modules using morphological operations 30,31 or the Hough transform.…”

“…Deep learning overcomes these problems and is applied to PV module detection by several works. 21,26,27…”

“…First works have shown promising results using deep learning for these tasks. 20,21 Following this recent trend, we use the Mask R-CNN instance segmentation framework 22 to robustly extract PV modules from thermal IR videos. A ResNet-50 deep convolutional classifier 23 is used for fine-grained classification of ten thermal anomalies.…”

Towards Fully Autonomous UAVs: A Survey

“…Other UAV applications include: solar panel inspection [ 266 , 267 ]; power line and tower inspections [ 268 , 269 ]; water quality monitoring [ 270 ]; magnetic field mapping [ 271 ]; load transportation [ 272 , 273 ]; contact inspection tasks [ 274 ]; road safety and traffic monitoring [ 275 , 276 , 277 ]; aerial photography and cinematography [ 278 ]; entertainment and aerial shows [ 279 , 280 , 281 , 282 ]; firefighting [ 283 , 284 , 285 ].…”