In view of the shortcoming of traditional image fusion based on discrete wavelet transform (DWT) with unclear textural information, an effective visible light and infrared image fusion algorithm via feature residual and statistical matching is proposed in this study. First, the source images are decomposed into low‐frequency coefficients and high‐frequency coefficients by DWT. Second, two different fusion schemes are designed for the low‐frequency coefficients and high frequency ones, respectively. The low‐frequency coefficients are fused by a local feature residual‐based scheme to achieve adaptive fusion; the high‐frequency coefficients are accomplished by a local statistical matching‐based scheme to extract the edge information effectively. Finally, the fused image is obtained by inverse DWT. Experimental results demonstrate that the proposed method can produce a more accurate fused image, leading to an improved performance compared with existing methods.