Automatic classification of sea ice and open water plays a vital role in climate change research, polar shipping, and other applications. Many deep learning-based methods are proposed to automatically classify sea ice and open water to address this issue. Even though these methods have achieved remarkable success, the noise phenomenon in SAR images still causes considerable limitations in the model performance.Meanwhile, these existing methods ignore multi-scale global information from large-scale SAR images which tends to produce misclassification. In this paper, we propose a novel Multi-scale Dual Attention Network (MSDA-Net) for the task. To tackle the first drawback, we introduce the information of relative position and high-pass filtering as two extra channels to reduce the noisy effects. Moreover, we propose a patch dual attention mechanism (PDAM) and embed it into the ConvNeXt blocks to capture the multi-channel and spatial features. To address the second problem, we propose a multi-scale spatial attention (MSSA) module to capture multi-scale global spatial information. The experiments show that the proposed method significantly outperforms state-of-the-art methods. In addition, comprehensive case studies are conducted, which verify the effectiveness of MSDA-Net in different SAR scenes.