Surface roughness is of great significance in maintaining mechanical performance and improving the reliability of the equipment. However, fast surface roughness evaluations that are sufficiently stable and efficient for engineering in-situ use have not yet been realized. To address this issue, an image-driven roughness intelligent method is proposed in this research. By evaluating the texture similarity intelligently between the testing image and the reference image, the surface roughness of the testing image can be acquired. Firstly, with a proposed adaptive texture extraction method, the texture feature of an image can be extracted even under a complex background. Secondly, by establishing the graph structure of the texture grayscale features, the similarity between different images is evaluated. Finally, by establishing a sparrow-optimized support vector machine regression method, the mapping relationship between the similarity and the surface roughness can be acquired. The experimental results indicate that the proposed method for intelligent evaluation of roughness has superior prediction performance (the average relative prediction error of Ra and Rz are 8.8156% and 8.0571%, respectively). Therefore, this work provides a useful tool for non-contact detection of workpiece surface roughness.