Real-Time Inspection System for Ballast Railway Fasteners Based on Point Cloud Deep Learning

Abstract

Rail fasteners are the most numerous components in railways and they should be inspected periodically. Manual inspection is currently a common solution, which is laborious and low-efficient. Some automatic inspection approaches are proposed. But for ballast railway fasteners inspection, debris, especially ballast along tracks may cover the fasteners, which is still a tricky problem. In this paper, a real-time inspection system for ballast railway fasteners based on point cloud deep learning is developed. Dense and precise point cloud of fastener is obtained from the structured light sensors in the system. The point cloud of fastener is segmented into different parts to avoid the interference of debris on fasteners. A ballast fastener point cloud semantic segmentation dataset is created based on automatic annotation method. Several deep learning point cloud segmentation models are tested in this dataset and PointNet++ is selected to be deployed in the real-time deep learning module of the system. Field tests on ballast railways show excellent accuracy and efficiency of this system.INDEX TERMS Ballast railway fasteners, fastener inspection, deep learning, neural network, point cloud semantic segmentation.

Semantic-Segmentation-Based Rail Fastener State Recognition Algorithm

“…For example, in [12], the authors proposed a template matching classification method to automatically collect and annotate fastener samples and further deployed a similarity-based deep convolutional neural network (DCNN) to estimate the fastener state. In [13], a real-time inspection system for ballast railway fasteners based on point cloud deep learning was developed, demonstrating excellent accuracy and efficiency in field testing on ballastless tracks. In [14], a fastener detection method based on visual rail inspection is proposed using material classification and semantic segmentation with DCNN to, respectively, identify and segment the different functional parts in a rail fastener image.…”

“…In formulas (13) and (14), T P represents the pixel sample data of the real fastener area located, and F P and F N , respectively, indicate the false and missed pixel data samples of the positioning experiment results relative to the theoretical railroad fastclip area in the truth map. In formula (15), the value of λ 2 is 0.3 [35].…”

“…To address the limitation of traditional detection methods, many automatic detection methods using machine vision and image processing technology have been proposed and achieved good experimental results [1][2][3][4][5][6][7][8][9][10][11]. Recently, deep learning theory has received increasing attention in target detection and image segmentation works and has been successfully applied to rail fastener detection [12][13][14]. However, these efforts are often heavily dependent on timeconsuming and expensive manual annotations.…”

Random-Forest Machine Learning Approach for High-Speed Railway Track Slab Deformation Identification Using Track-Side Vibration Monitoring

“…In recent years, machine learning methods have had many applications in track defect identification and prediction. Machine learning methods were used to detect defects of important components of the track structure that are directly related to safe train operation, including the defects of rails [39,40], fasteners [41][42][43][44], rail pads [45], and turnout [46]. The classification of three types of rail defects-surface defect, cross level defect and depression in track profiles-by track geometry data on the basis of logistic regression and decision tree [39] and the classification of rail crack by acoustic emission waves on the basis of a multibranch convolutional neural network (CNN) [40] were explored.…”

“…The classification of three types of rail defects-surface defect, cross level defect and depression in track profiles-by track geometry data on the basis of logistic regression and decision tree [39] and the classification of rail crack by acoustic emission waves on the basis of a multibranch convolutional neural network (CNN) [40] were explored. The rail fastener defects were detected from images on the basis of a CNN [41], generative adversarial network, residual network [42], point cloud deep learning [43], and Faster region-CNN [44]. The dynamic stiffness of rail pads was predicted using several machine learning methodologies (multilinear regression, K nearest neighbors, regression tree, random forest, gradient boosting, multilayer perceptron, and support vector machine (SVM)) [45].…”