A Survey on Audio-Video Based Defect Detection Through Deep Learning in Railway Maintenance

Abstract

Within Artificial Intelligence, Deep Learning (DL) represents a paradigm that has been showing unprecedented performance in image and audio processing by supporting or even replacing humans in defect and anomaly detection. The Railway sector is expected to benefit from DL applications, especially in predictive maintenance applications, where smart audio and video sensors can be leveraged yet kept distinct from safety-critical functions. Such separation is crucial, as it allows for improving system dependability with no impact on its safety certification. This is further supported by the development of DL in other transportation domains, such as automotive and avionics, opening for knowledge transfer opportunities and highlighting the potential of such a paradigm in railways. In order to summarize the recent state-of-the-art while inquiring about future opportunities, this paper reviews DL approaches for the analysis of data generated by acoustic and visual sensors in railway maintenance applications that have been published until August 31st, 2021. In this paper, the current state of the research is investigated and evaluated using a structured and systematic method, in order to highlight promising approaches and successful applications, as well as to identify available datasets, current limitations, open issues, challenges, and recommendations about future research directions.

Deep Learning for Automated Visual Inspection in Manufacturing and Maintenance: A Survey of Open- Access Papers

“…Zheng et al [12], e.g., focused their survey on surface defect detection with DL techniques. The same applies to the surveys by Jenssen et al [13], Sun et al [14], Yang et al [15], Nash et al [16], Liu et al [17], and Donato et al [18], who concentrated on different industry sectors, like steel production, railway applications, power lines, or manufacturing in general. There are also many publications in the areas of civil engineering and structural health monitoring.…”

Self-adapting reconfigurable multiply-accumulator for real-time image processing in embedded systems

“…This is particularly important in resource-constrained devices, where area and power savings must be taken into serious consideration, 2 . 3 Such effort has been exacerbated in recent years by the widespread adoption of computationally intensive Machine Learning (ML) methods 4 in various applications, including but not limited to medical imaging, [5][6][7] image and video enhancement, 8,9 image segmentation, 10 defect detection, 11,12 person re-identification, 13 and remote sensing. [14][15][16] Many of these applications rely on Convolutional Neural Networks (CNNs) and digital filters, which require the parallel processing of several convolutions 1,2,17 for which the use of dedicated Hardware (HW) accelerators appears to be the only viable solution to meet the throughput requirements.…”