Pavement Defect Segmentation in Orthoframes with a Pipeline of Three Convolutional Neural Networks

Abstract

In the manuscript, the issue of detecting and segmenting out pavement defects on highway roads is addressed. Specifically, computer vision (CV) methods are developed and applied to the problem based on deep learning of convolutional neural networks (ConvNets). A novel neural network structure is considered, based on a pipeline of three ConvNets and endowed with the capacity for context awareness, which improves grid-based search for defects on orthoframes by considering the surrounding image content—an approach, which essentially draws inspiration from how humans tend to solve the task of image segmentation. Also, methods for assessing the quality of segmentation are discussed. The contribution also describes the complete procedure of working with pavement defects in an industrial setting, involving the workcycle of defect annotation, ConvNet training and validation. The results of ConvNet evaluation provided in the paper hint at a successful implementation of the proposed technique.

What's Cracking? A Review and Analysis of Deep Learning Methods for Structural Crack Segmentation, Detection and Quantification

“…The Intersection over Union metric (IoU ) is another metric that scores the overlap between a prediction and a ground truth. It is often used for detection and segmentation tasks [19], [34], [81], [120]. Let X be a predicted output and Y the ground truth, then IoU is calculated as follows:…”

Automated Multi-Type Pavement Distress Segmentation and Quantification Using Transformer Networks for Pavement Condition Index Prediction

“…However, pavements do not have only one type of distress, and each type of distress has its own exclusive characteristics and apparent forms. For this reason, Lõuk et al [38] applied a U-Net-like network architecture with different levels of contextual resolution to integrate more contextual information. The authors considered different types of pavement distress, but they only extracted the planar regions of the different types of distress.…”

Special Issue on Bio-Inspired Algorithms for Image Processing

“…Last but not least, in the paper of Roland Lõuk et al, titled Pavement Defect Segmentation in Orthoframes with a Pipeline of Three Convolutional Neural Networks [6], a novel ensemble neural network structure is proposed for highway pavement defect detection. The input data are masked orthoframes, which are processed in a pipeline of three models: road segmentation is followed by defect detection and, finally, defect segmentation.…”