The performance of a speaker verification system is damagingly affected by large amount of noise. In order to compensate the mismatch between enrollment and test acoustic conditions, this paper presents a novel approach based on Gaussian Mixture Model-Universal Background Model (GMM-UBM) algorithm. Noisy background adaption is proposed to make speaker models more close to the one in real-world scenarios. Since there is great difference among various noise types, noise recognition is applied to help choose appropriate background model in accordance with certain noisy condition. Moreover, a Mel-domain de-noise method is used as a suitable noise reduction front-end. Finally, experiments on corrupted TIMIT database showed that the proposal could reduce up to 26.38% and an average of 16.44% equal error rate (EER) compared to the baseline, indicating its advantages on speaker verification under various noisy conditions.