Use of deep convolutional neural networks and change detection technology for railway track inspections

Abstract

Railroad track inspections conducted in accordance with federal regulations and internal railway operating practices result in significant labor costs and occupy valuable network capacity. These factors, combined with advancements in the field of machine vision, have encouraged a transition from human visual inspections to machine-based alternatives. Commercial machine vision technologies for railway inspection currently exist, and automated analysis approaches—which deliver objective results—are available in some systems. However, they are limited to a “pass/fail” approach through the detection of components which fail to meet maintenance or geometry thresholds, as opposed to being able to detect subtle changes in track conditions to identify evolving problems. To overcome these limitations, this paper presents results from the field deployment and validation of a system that pairs three-dimensional (3D) machine vision with automated change detection technology. The change detection approach uses a deep convolution neural network (DCNN) to accurately characterize track conditions between repeat runs. Current automated track inspection technologies were studied, and the applicability of change detection is discussed. The paper presents the process for 3D image capture, DCNN training, and evaluation by comparing DCNN results to an expert human evaluator. Finally, it presents change detection results for fastener presence and spike height. Results indicate that this technology can successfully identify fasteners and spikes with percent accuracies greater than 98% and that it can successfully generate change detection results for comparison of track condition among runs.

Development of track component health indices using image-based railway track inspection data

“…This was accomplished by first conducting a ground truth survey, visually checking the images and comparing findings to the results output by the DCNNs to data collected by human inspectors on the ground. Further discussion on data validation was documented by Harrington et al 13…”

“…These images are subsequently evaluated by the DCNNs which identify various attributes related to the health of the track system and its components. The LRAIL system and its attributes were described in greater detail by Fox-Ivey et al 12,14 and Harrington et al 13 The depth and breadth of the data collected by LRAIL present an opportunity to process and present information to a variety of end users within the railway organizational structure. Distinct end-users have disparate use cases for such data and need to consume it at a different level of specificity.…”

“…The introduction of autonomous inspection systems leveraging three-dimensional (3D) laser-based triangulation and deep convolutional neural networks (DCNNs) can provide a solution to inventory, track, and optimize rail infrastructure assets. 12,13 When 3D laser technology is paired with properly trained DCNNs a variety of track components and conditions can be reliably detected. 13 The research described here within uses a 3D laser triangulation scanning system which combines pulsed, high-power, invisible laser line projectors and synchronized cameras to capture a high-resolution intensity image and range profile of rail track.…”

“…12,13 When 3D laser technology is paired with properly trained DCNNs a variety of track components and conditions can be reliably detected. 13 The research described here within uses a 3D laser triangulation scanning system which combines pulsed, high-power, invisible laser line projectors and synchronized cameras to capture a high-resolution intensity image and range profile of rail track. 14 When track components are analyzed as a collective system, the output provides a holistic view of the track condition and gives insight to future problems the track may experience.…”

“…After multiple runs and training of the algorithm, 98% precision was achieved. 12,13 Additional information on the development of the DCNN and initial training data generated were described by Harrington et al 13 Additionally, Harrington et al 13 documented the development of a proposed methodology to report inspection data and established a benchmark for uniform representation of 3D laser scanning data using strip charts.…”