A railway track reconstruction method using robotic vision on a mobile manipulator: A proposed strategy

Rahman, Miftahur; Liu, Haochen; Masri, Mohammed; Durazo-Cardenas, Isidro Sergio; Starr, Andrew

doi:10.1016/j.compind.2023.103900

Cited by 16 publications

(4 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, the railway track detection and 3D fitting aims to obtain the spatial coordinates of the tracks to determine their accurate 3D trajectory. It should be noted that we do not intend to model the detailed size and external shape of the tracks [34,35]. Therefore, a two-stage method for railway track detection in the image is proposed.…”

Section: Railway Track Detection In the Imagementioning

confidence: 99%

A Real-Time Method for Railway Track Detection and 3D Fitting Based on Camera and LiDAR Fusion Sensing

Tang,

Cao,

Yang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

Railway track detection, which is crucial for train operational safety, faces numerous challenges such as the curved track, obstacle occlusion, and vibrations during the train’s operation. Most existing methods for railway track detection use a camera or LiDAR. However, the vision-based approach lacks essential 3D environmental information about the train, while the LiDAR-based approach tends to detect tracks of insufficient length due to the inherent limitations of LiDAR. In this study, we propose a real-time method for railway track detection and 3D fitting based on camera and LiDAR fusion sensing. Semantic segmentation of the railway track in the image is performed, followed by inverse projection to obtain 3D information of the distant railway track. Then, 3D fitting is applied to the inverse projection of the railway track for track vectorization and LiDAR railway track point segmentation. The extrinsic parameters necessary for inverse projection are continuously optimized to ensure robustness against variations in extrinsic parameters during the train’s operation. Experimental results show that the proposed method achieves desirable accuracy for railway track detection and 3D fitting with acceptable computational efficiency, and outperforms existing approaches based on LiDAR, camera, and camera–LiDAR fusion. To the best of our knowledge, our approach represents the first successful attempt to fuse camera and LiDAR data for real-time railway track detection and 3D fitting.

show abstract

Section: Railway Track Detection In the Imagementioning

confidence: 99%

A Real-Time Method for Railway Track Detection and 3D Fitting Based on Camera and LiDAR Fusion Sensing

Tang,

Cao,

Yang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…The attached RGB-D camera can provide 3D image of the target object, but it fails to capture information from the side and back of the object (Figure 18a). A 3D reconstruction method has been proposed and demonstrated with RIRS and sensor fusion technology to improve the track inspection technique as the reconstructed 3D model (Figure 18b) provides more dimensional and textural information than RGB-D camera output (Figure 18a) [150]. Additionally, the 3D reconstructed model can be fused with depth camera to provide better robot perception, with the LiDAR point cloud for environmental sensing and GPS data for geo-location.…”

Section: Case Studymentioning

confidence: 99%

“…Adopting train services for public Though there are some industrial 3D imaging solutions such as Zivid 2 [151], they are more expensive than monocular cameras. The method proposed in [150] was a costeffective solution as it created 3D model of the target object using a monocular camera and open source structure-from-motion (SfM) technique [152]. Moreover, because of the flexibility of the overall method, this technique can be used for creating digital model of the railway track without human involvement, as shown in Figure 18c, which can be beneficial for creating a digital twin of railway tracks, performing digital surveys or creating railway datasets for machine learning algorithms.…”

Section: Benefits Of Using a Mobile Manipulator For Railway Track Mai...mentioning

confidence: 99%

A Review on the Prospects of Mobile Manipulators for Smart Maintenance of Railway Track

et al. 2023

Self Cite

View full text Add to dashboard Cite

Inspection and repair interventions play vital roles in the asset management of railways. Autonomous mobile manipulators possess considerable potential to replace humans in many hazardous railway track maintenance tasks with high efficiency. This paper investigates the prospects of the use of mobile manipulators in track maintenance tasks. The current state of railway track inspection and repair technologies is initially reviewed, revealing that very few mobile manipulators are in the railways. Of note, the technologies are analytically scrutinized to ascertain advantages, unique capabilities, and potential use in the deployment of mobile manipulators for inspection and repair tasks across various industries. Most mobile manipulators in maintenance use ground robots, while other applications use aerial, underwater, or space robots. Power transmission lines, the nuclear industry, and space are the most extensive application areas. Clearly, the railways infrastructure managers can benefit from the adaptation of best practices from these diversified designs and their broad deployment, leading to enhanced human safety and optimized asset digitalization. A case study is presented to show the potential use of mobile manipulators in railway track maintenance tasks. Moreover, the benefits of the mobile manipulator are discussed based on previous research. Finally, challenges and requirements are reviewed to provide insights into future research.

show abstract

“…These inspection robots are equipped with a myriad of sensors, encompassing RGB and infrared (IR) imaging, auditory capabilities, Light Detection and Ranging (LiDAR) technology, and gas hazard sensors [ 3 , 4 , 5 , 6 , 7 ].…”

Section: Introductionmentioning

confidence: 99%

Advanced Image Analytics for Mobile Robot-Based Condition Monitoring in Hazardous Environments: A Comprehensive Thermal Defect Processing Framework

Siami,

Barszcz,

Zimroz

2024

Sensors

View full text Add to dashboard Cite

In hazardous environments like mining sites, mobile inspection robots play a crucial role in condition monitoring (CM) tasks, particularly by collecting various kinds of data, such as images. However, the sheer volume of collected image samples and existing noise pose challenges in processing and visualizing thermal anomalies. Recognizing these challenges, our study addresses the limitations of industrial big data analytics for mobile robot-generated image data. We present a novel, fully integrated approach involving a dimension reduction procedure. This includes a semantic segmentation technique utilizing the pre-trained VGG16 CNN architecture for feature selection, followed by random forest (RF) and extreme gradient boosting (XGBoost) classifiers for the prediction of the pixel class labels. We also explore unsupervised learning using the PCA-K-means method for dimension reduction and classification of unlabeled thermal defects based on anomaly severity. Our comprehensive methodology aims to efficiently handle image-based CM tasks in hazardous environments. To validate its practicality, we applied our approach in a real-world scenario, and the results confirm its robust performance in processing and visualizing thermal data collected by mobile inspection robots. This affirms the effectiveness of our methodology in enhancing the overall performance of CM processes.

show abstract

A railway track reconstruction method using robotic vision on a mobile manipulator: A proposed strategy

Cited by 16 publications

References 43 publications

A Real-Time Method for Railway Track Detection and 3D Fitting Based on Camera and LiDAR Fusion Sensing

A Real-Time Method for Railway Track Detection and 3D Fitting Based on Camera and LiDAR Fusion Sensing

A Review on the Prospects of Mobile Manipulators for Smart Maintenance of Railway Track

Advanced Image Analytics for Mobile Robot-Based Condition Monitoring in Hazardous Environments: A Comprehensive Thermal Defect Processing Framework

Contact Info

Product

Resources

About