Winter road surface condition classification using convolutional neural network (CNN): visible light and thermal image fusion

Zhang, Ce; Nateghinia, Ehsan; Miranda-Moreno, Luis F.; Sun, L.

doi:10.1139/cjce-2020-0613

Cited by 8 publications

(8 citation statements)

References 20 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, the experimental comparison suggests that the network performance is the best when the segment value is 3. Therefore, this work divides the video into three segments [28].…”

Section: B Comparison Of Different Video Segmentation and Fusion Methodsmentioning

confidence: 99%

The Application of Deep Convolution Neural Network in Volleyball Video Behavior Recognition

Liang

2022

IEEE Access

View full text Add to dashboard Cite

The purpose is to solve the problems of solid subjectivity in the traditional manual analysis of volleyball game videos and the traditional Human Behavior Recognition (HBR) algorithm, such as excessive calculation, high hardware conditions, and poor long-stream video modeling ability. Firstly, this paper expounds on the relevant theories. Secondly, a fusion Convolutional Neural Network (CNN)-based HBR model is implemented by combining two-stream CNN (TSCNN), Three-Dimensional (3D) CNN, and Long Short-Term Memory (LSTM) network. Notably, the LSTM has an excellent long-term Dynamic Information Extraction (DIE) ability. Finally, the public dataset is selected to verify the model's volleyball-game-videooriented HBR performance. The experimental results corroborate that: (1) The parameters of the proposed fusion-CNN-based HBR model are determined as follows: the number of video segments is three, the average method is used for feature fusion, and then the HBR accuracy is the highest when the fusion ratio of spatial feature map and temporal feature map is 4:6, and the learning rate is 0.0014. (2) The average HBR accuracy of the proposed fusion-CNN-based HBR model on three different datasets is 4%, 2.7%, and 3% higher than that of other popular networks, respectively. The improvement effect of the model is remarkable, and it is suitable for the study of Human Behavior Analysis (HBA) in volleyball match videos. Finally, the proposed HBR model can provide more accurate results for volleyball video-based HBR, which is significant in promoting the rapid development of volleyball sports.

show abstract

“…Thus, the experimental comparison suggests that the network performance is the best when the segment value is 3. Therefore, this work divides the video into three segments [28].…”

Section: B Comparison Of Different Video Segmentation and Fusion Methodsmentioning

confidence: 99%

The Application of Deep Convolution Neural Network in Volleyball Video Behavior Recognition

Liang

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Of the studies discussed thus far, the images used for model development have all been conventional images. Straying away from this trend, Nateghinia et al ( 7 ) adopted use of infrared (IR) images as part of their model development process. To collect image data, an IR and a conventional camera were both mounted to the front of the vehicle.…”

Section: Convolutional Neural Network and Related Workmentioning

confidence: 99%

“…Overall, as seen from the literature discussed in this section, previous studies on the topic of winter RSC are centered around developing RSC classification models for rural highways; only one study— Nateghinia et al ( 7 )—developed an urban model, but it was designed for contaminant identification, not for RSC classification. Furthermore, in respect of model development itself, previous studies either developed their own original CNN model from scratch or focused on repurposing a pre-trained ImageNet model.…”

Section: Convolutional Neural Network and Related Workmentioning

confidence: 99%

“…Many researchers around the world have since then worked on developing CNN models that are calibrated specifically to classify winter RSC, including both creating a custom CNN model and taking pre-trained CNN models designed for other purposes and tailoring them to make RSC classifications (4)(5)(6)(7)(8). While some notable methodological advancements have been made to date, the study area used in these previous studies focuses on major highways in the rural context.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Development of a Highly Transferable Urban Winter Road Surface Classification Model: A Deep Learning Approach

Xie

Kwon

2022

Transportation Research Record: Journal of the Transportation R

View full text Add to dashboard Cite

Road surface condition (RSC) is an important performance indicator for winter road maintenance personnel to maintain safe driving conditions. This becomes more apparent in inclement weather events where timely clearing of snow is highly prioritized. Considering the vast road networks that need to be covered, many transportation agencies have been using camera images to view real-time RSC directly; however, monitoring conditions via these cameras is still being done manually, thereby hindering its full utilization for optimizing maintenance services. Many studies have attempted to develop a deep-learning-based approach known as convolution neural network (CNN) to automate the process of RSC image classification. When implemented, RSC can be extracted from road images without human involvement. However, efforts made thus far have been focused on rural highways, with performance in the urban context being the least explored. Furthermore, CNN models developed in previous studies have been trained either from scratch or via transfer learning, but only a few studies have investigated transfer learning using a pre-trained RSC model. To address these gaps, an urban RSC model was developed in this study via transfer learning using a pre-trained RSC CNN model. The image dataset used contains 3914 urban images collected in a residential area south of Edmonton, Alberta. With these images, the pre-trained RSC model was fine-tuned via transfer learning and underwent hyperparameter optimization to boost performance further, yielding a high classification accuracy of 98.21% and an F1-score of 98.4%, which surpassed the accuracy of the model trained from scratch.

show abstract

“…However, with the growing number of cameras installed in road networks, multiple deep learning models are trained using image datasets produced by these cameras. Zhang et al [4] introduced a visible and infrared image fusion method for the road snow cover classification. A recent work studied the use of webcams installed on interstate roads to estimate weather and surface conditions [5].…”

Section: Introductionmentioning

confidence: 99%

A Computer Vision-Based Framework for Snow Removal Operation Routing

Karaa,

Ghazzai,

Massoud

et al. 2024

IEEE Open J. Circuits Syst.

View full text Add to dashboard Cite

During snowfall, the utility of the road infrastructure is critical. Roads must be effectively cleared to ensure access to important locations and services. In this paper, we present an end-to-end framework for snow removal vehicle routing based on road priority. We offer an artificial intelligence-based image-based approach for estimating snow depth and traffic volume on roads. For segments monitored by CCTV cameras, we exploit images and supervised learning models to perform this task. For unmonitored roads, we use the Graph Convolutional Network architecture to predict parameters in a semi-supervised manner. Following that, we assign priority weights to all graph edges as a function of image-based attributes and road categories. We test the method using a real-world example, simulating snow removal within a study area in Montreal, Quebec, Canada. As input for the framework, we collect CCTV image data and combine it with a 2D map. As a result, more efficient snow removal operation can be achieved by optimizing the trajectories of trucks based on the computer vision module outputs.INDEX TERMS Snow removal, road service prioritization, graph analytics, computer vision, vehicle routing.

show abstract

Winter road surface condition classification using convolutional neural network (CNN): visible light and thermal image fusion

Cited by 8 publications

References 20 publications

The Application of Deep Convolution Neural Network in Volleyball Video Behavior Recognition

The Application of Deep Convolution Neural Network in Volleyball Video Behavior Recognition

Development of a Highly Transferable Urban Winter Road Surface Classification Model: A Deep Learning Approach

A Computer Vision-Based Framework for Snow Removal Operation Routing

Contact Info

Product

Resources

About