Aiming at effective outlier elimination in the biological near-infrared spectral and achieving high accuracy predictive modeling, this paper proposes a novel outlier elimination method based onX-Yvariance and leverage analysis. Firstly, the characters of near-infrared spectral are summarized; then residual sampleX-variance, leverage, and residual sampleY-variance are concatenated as a divergence measurement. We further compared the proposed method withX-Yvariance, Mahalanobis distance, and HotellingT2statistical analysis; the experiment results demonstrate that the proposed methods have competitive outlier elimination and better performance in time complexity and accuracy. The proposed method can also be adopted for other outlier elimination tasks.