Identification of extremely similar animal fibers has always been one of the challenging research topics in the textile field. In this paper, an improved image-based recognition method was proposed for the identification of extremely similar animal fibers including cashmere and wool. A total of 100 groups of wool and 100 groups of cashmere fiber images were collected using the self-developed microscope image analysis system. The contrast of original fiber micro images usually was not high enough and some impurities always existed during the slicing process, so the matched filters were first applied for these images to extract the enhanced fiber binary texture, which only needs to set reasonable segment length and the threshold for different fiber images to eliminate impurities and background. Then, the high-dimensional texture features of the original color images, gray images, and the images processed by matched filter were extracted by calculating and analyzing the histogram of oriented gradient (HOG). The 200 sets of original color images, gray images, and processed images were divided into the training set and testing set according to different proportions, and the recognition expert system based on the support vector machine (SVM) could be trained and validated accordingly. The experimental results show that the recognition accuracy of the fiber images processed by matched filter was obviously improved compared with that of the other two data sets, and the recognition rate reaches the highest with 92.5%. It also proves that the proposed algorithm in this paper can classify and identify extremely similar wool and cashmere fibers more quickly and effectively compared with other texture feature extraction and identification algorithms. INDEX TERMS Extremely similar fibers, image identification, matched filter, HOG, SVM.