Annular linear polarized light is used as the illumination source of the reflective dark-field detecting system in this paper. According to the theories of the Bidirectional Reflectance Distribution Function (BRDF) and multi-beam interference, the influence of the light polarization state on the intensity distribution of the scattering light is analyzed in detail. For surface defects, a simulation model of dark-field imaging is established based on the Finite-Difference Time-Domain method (FDTD). Both the near-field and the far-field scattering intensity distribution caused by surface defects are calculated under different illumination conditions. The incidence angle and polarization state of illumination light are optimized. Simulation and experimental results show that the image quality will be minimally affected by the interference effect while P-polarized light illuminates with the incident angle of 45°. The higher measurement accuracy of the dark-field imaging detection system can be obtained when the optimized illumination scheme is used.