Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces with a Recurrent Neural Network

100

Autonomous vehicle (AV) stakeholders continue to seek assurance of the safety performance of this new technology through AV testing on in‐service roads, AV‐dedicated road networks, and AV test tracks. However, recent AV‐related fatalities on in‐service roads have exacerbated public skepticism and eroded some public trust in the safety of AV operations. Further, test tracks are unable to characterize adequately the real‐world driving environment. For this reason, driving simulators continue to serve as an attractive means of AV testing. However, in most AV driving simulators, the AV operation is based on commands external to the vehicle and embedded in the code for the driving environment. To address the simulation shortfalls associated with this approach, this paper develops a deep convolutional neural network–long short‐term memory (CNN–LSTM) algorithm for self‐driving simulation. This algorithm observes and characterizes the AV's driving environment, and controls the AV movement in the driving simulation. The CNN part extracts features that use transfer learning to introduce human prior knowledge, and the LSTM part uses temporal information to process the extracted features, and incorporates temporal dynamics to predict driving decisions. The AV may also use an external server with a database containing road environment data as an additional source of information. It is acknowledged that different driving simulators differ in their functions and their capabilities to access driving‐environment data. Therefore, to make it sufficiently flexible to facilitate replication by other researchers that use driving simulators, the algorithm has been designed and demonstrated using only image data of the driving environment as input. This is because roadway image data are easily and readily accessible from the screen of any driving simulator. The proposed algorithm was tested using the open racing car simulator test track platform and was found to be able to mimic human driving decisions with a high degree of accuracy.

Section: Methodsmentioning

confidence: 99%

Section: Deep Neural Networkmentioning

confidence: 99%

A deep learning algorithm for simulating autonomous driving considering prior knowledge and temporal information

Chen

Leng

Labi

2019

100

“…Zhang et al. () proposed a recurrent neural network (CrackNet‐R) for pixel‐level pavement crack detection. Compared with the original CrackNet, CrackNet‐R performs better in terms of speed and detection accuracy.…”

Section: Related Workmentioning

confidence: 99%

“…For processing high-resolution road images, a computationally more efficient crack detection model is essential. Zhang et al (2018a) proposed a recurrent neural network (CrackNet-R) for pixel-level pavement crack detection. Compared with the original CrackNet, CrackNet-R performs better in terms of speed and detection accuracy.…”

Section: Deep Learning-based Crack Detectionmentioning

confidence: 99%

Encoder–decoder network for pixel‐level road crack detection in black‐box images

Bang

Park

Kim

et al. 2019

341

170

Timely monitoring of pavement cracks is essential for successful maintenance of road infrastructure. Accurate information concerning crack location and severity enables proactive management of the infrastructure. Black‐box cameras, which are becoming increasingly widespread at an affordable price, can be used as efficient road‐image collectors over a wide area. However, the cracks in these images are difficult to detect, because the images containing them often include objects other than roads. Thus, we propose a pixel‐level detection method for identifying road cracks in black‐box images using a deep convolutional encoder–decoder network. The encoder consists of convolutional layers of the residual network for extracting crack features, and the decoder consists of deconvolutional layers for localizing the cracks in an input image. The proposed network was trained on 427 out of 527 images extracted from black‐box videos and tested on the remaining 100 images. Compared with VGG‐16, ResNet‐50, ResNet‐101, ResNet‐200 with transfer learning, and ResNet‐152 without transfer learning, ResNet‐152 with transfer learning exhibited the best performance, achieving recall, precision, and intersection of union of 71.98%, 77.68%, and 59.65%, respectively. The experimental results prove that the proposed method is optimal for detecting cracks in black‐box images at the pixel level.

“…Machine learning nonparametric modeling (Adeli & Hung, 1994;Marsland, 2011;Reich & Barai, 1999) has attracted increasing attention in various engineering disciplines Rafiei & Adeli, 2018;Rafiei, Khushefati, Demirboga, & Adeli, 2017;Xue & Li, 2018;Yang et al, 2018;Zhang et al, 2019). It provides a powerful toolkit to establish spatial model based on the unstructured data (Kanevski, Timonin, & Pozdnukhov, 2009;Karpatne, Ebert-Uphoff, Ravela, Babaie, & Kumar, 2018).…”

Section: Introductionmentioning

confidence: 99%

Broad learning for nonparametric spatial modeling with application to seismic attenuation

Kuok

Yuen

2019

Spatial modeling is a core element in geographical information science. It incorporates geographic information to construct the relationship for interpreting the behavior of spatial phenomena. In this paper, a broad learning framework for nonparametric spatial modeling is presented. Broad learning overcomes the obstacle of expensive computational consumption in deep learning and provides a powerful computationally efficient alternative. In contrast to the deep learning architecture that is configured with stacks of hierarchical layers, broad learning networks are established in a flat manner that can be flexibly reconfigured with the inherited information from the trained network. To develop the broad learning network, a simple prototype network is established as the initial trial and it is modified incrementally to enhance its data fitting capacity. Consequently, complex relationship of unstructured spatial data can be modeled efficiently. To demonstrate the efficacy and applicability of the broad learning framework, we will present a simulated example and a real application using the strong ground motion records on the 2008 great Wenchuan earthquake.