Human hearing is robust to noise, but the basis of this robustness is poorly understood. We explored whether internal models of environmental noise structure aid the perception of sounds in noise. One prediction of this hypothesis is that hearing should improve with exposure to a noise source, since noise properties can be better estimated with more samples. Consistent with this idea, we found that detection, recognition, and localization in real-world background noise improved with exposure to the background. A model that detected outliers from a distribution of background noise accounted for this pattern of performance. However, human performance was additionally enhanced for recurring backgrounds and was robust to interruptions in the background, suggesting listeners build up and maintain representations of noise properties over time. The results suggest noise robustness is supported by internal models—noise schemas—that capture the structure of noise and are used to estimate other concurrent sounds.