Enhancement of Traffic Images Under Different Weather Conditions Using Pynet

Abstract

Convolutional neural networks have real practical application potentials, such as for autonomous driving and semantic segmentation, but they are known to be sensitive to image degradations and corruption. Over the years, image enhancement in deep learning has shown drastic improvement. However, the clarity and quality of images still badly affect the robustness of semantic segmentation, especially for traffic images under different weather conditions. This paper proposes image enhancement performance using PyNET deep learning methods for more robust semantic segmentation of traffic images under different conditions. This work also proposes a new metric to objectively estimate the performance known as Image Similarity Metrics (ISM). Modification to PyNET is made in this work to allow this deep learning model to be used to enhance traffic images under various weather conditions such as fog, night, and rain. We compared PyNET performance against the Deep Convolutional Networks (DPED) and the Cycle Generative Adversarial Networks (CycleGAN) to evaluate the improvement gained by these image enhancement methods. Based on our experiment, PyNET gives the best image enhancement performance among those three methods in all weather conditions according to our proposed ISM. To support the validity of the ISM result, we performed tests by semantic segmentation of traffic images using ResNet-18 on the PyNET-enhanced images. Based on semantic segmentation results, PyNET improves the semantic segmentation of traffic images under different weather conditions by as much as 17% accuracy and delivers performance that directly validates the ISM scores, by showing that PyNET delivers the best semantic segmentation improvement in fog and night images