RSAN: A Retinex based Self Adaptive Stereo Matching Network for Day and Night Scenes

Abstract

It is essential in many robot tasks to retrieve depth information, while it still remains a challenging problem to get robust depth in unfavorable conditions such as night or rainy environments. With the development of convolutional neural networks (CNNs), a large number of algorithms have emerged to tackle the problem of dark image enhancement and depth estimation, but there are few works focus on recovering depth map in dark environments and normal light condition. To meet this demand, we proposed a neural network which takes the paired stereo images in all light conditions as input and estimates the fully scaled depth map. The network contains a novel feature extractor and a stereo matching module which follows a light-weight manner to guarantee this work practical for real robotic applications. We introduced the Retinex Theory into depth estimation and trained the decomposition module with LOL dataset. Then it is adapted into depth estimation by fusing the decompose module into stereo matching algorithm. The whole network is then trained in an end-to-end manner. To demonstrate the robustness and effectiveness of our proposed method, we perform various studies and compare our results to the state-of-the-art algorithms in depth estimation as well as direct combination of image enhancement and stereo matching algorithm. We also collect stereo images in real night environments and present the improved performance of our network.