Railroad Crossing Surveillance and Foreground Extraction Network: Weakly Supervised Artificial-Intelligence Approach

Abstract

According to the report to Congress National Strategy to Prevent Trespassing on Railroad Property issued by the Federal Railroad Administration, trespassing is currently the number one cause of all railroad-related deaths. The number of fatalities resulting from trespassing, including both illegally entering and remaining in the railroad right-of-way, is even higher than the number of deaths caused by collisions between vehicles and trains. Accurately and effectively detecting objects at rail crossings is critical for improving railway safety. The convolutional neural network (CNN) is one of the most potent tools in image processing and video surveillance. However, the current CNNs under supervised learning rely on a substantial amount of manually labeled images, which is labor-intensive and time-consuming. This study proposes a weakly supervised learning technique for training foreground-segmentation networks and develops a tailored CNN network, RC-SAFE (railroad crossing surveillance and foreground extraction), to accurately and effectively detect and track both still and moving objects in the railroad crossing area. The weakly supervised learning significantly reduces the load for image labeling. The experimental results on the public CDnet 2014 dataset show that the weakly supervised network outperforms the supervised network and some other state-of-the-art foreground-segmentation methods. Railroad crossing monitoring videos are collected at different railroad crossings to validate the proposed RC-SAFE network. The results confirm the robustness of the proposed RC-SAFE network, and that RC-SAFE can detect any object that does not belong to the railroad crossing, which outperforms object-detection-based models, such as Mask RCNN (a region-based CNN).

A Systematic Review of Railway Trespassing: Problems and Prevention Measures

“…Trespasser profile and behaviour (Silla and Luoma, 2012) [43], (Silla and Luoma, 2009) [11], (Skladana et al, 2019) [40], (Pelletier, 1997) [13], (Patterson, 2004) [24], (Federal Railroad Administration, 2008) [30], (Davis et al, 1997) [56], (Matzopoulos et al, 2006) [71], (Havârneanu et al, 2016) [72], (Havârneanu et al, 2015) [16], (Hoekstra and Wegman, 2011) [73], (Waterson et al, 2015) [74], (DaSilva and Carroll, 2011) [75], (Department of transportation (FRA), 2011) [76], (Edwards, 1953) [77], (Cohen et al, 2003) [78], (Ricketts et al, 2010) [79], (Ricketts, 2015) [80], (Lobb et al, 2003) [81], (Hallewell et al, 2022) [82] 2. Signalization, technological and technical measures for trespassing prevention (Silla and Luoma, 2011) [83], (Federal Railroad Administration 2011) [84], (Waterson and Monk, 2014) [85], (Horton and Da Silva) [86], (Oh et al, 2022) [87], (Gao et al, 2021) [88], (Tang et al, 2023)…”

“…The objective of the experiment was to limit the reliance on a manually labelled image with weakly supervised learning, which in turn reduces the load of image labelling. The results were more accurate and effective in the detection of moving and still objects on railway tracks [89]. The surveillance data system, ARTS, for trespassing detection also uses CNN deep learning architecture in two steps: (1) the detection of trespassing activity and (2) highreliability classification using a deep neural network.…”