A multimodal detection system with complementary capabilities for efficient detection was developed for impurity detection. The system consisted of a visible light camera, a multispectral camera, image correction and registration algorithms. It can obtain spectral feature and color feature at the same time, and has higher spatial resolution than a single spectral camera. This system was applied to detect impurities in Pu ’er tea to verify its high efficiency. The spectral and color features of each pixel in the images of Pu ’er tea were obtained by this system and used for pixel classification. The experimental results show that the accuracy of Support Vector Machine (SVM) model based on combined features is 93%, which is 7% higher than that based on only spectral features. By applying median filtering algorithm and contour detection algorithm to the label matrix extracted from pixel-classified images, 8 impurities except hair were detected successfully. Moreover, taking advantage of the high resolution of visible light camera, small impurities can be clearly imaged. By comparing the segmented color image with the pixel-classified image, small impurities such as hair could be detected successfully. Finally, it is proved that the system can obtain multiple images to allow a more detailed and comprehensive understanding of the detected items, and has excellent ability to detect small impurities.