Image coloring is the process of adding color to grayscale images with the assistance of a computer. This paper proposes an optimal coloring algorithm based on non-local neighborhood search to better deal with boundary color penetration when colorizing grayscale images. First, the K non-local texture most similar neighborhoods are searched based on the image texture features. Then, based on the neighborhood pixels with similar intensity having similar color, the group of similar neighborhood pixels is further searched. The search results take into account not only the pixel texture features but also the light intensity distribution. Finally, the neighborhood pixels are introduced into the optimized coloring method, and the image is colored. The experimental results show that the way can effectively reduce boundary color penetration by non-local propagation of color information. In the case of only a tiny amount of color smearing, the method in this paper can provide better visual effects.