Classification of material type is crucial in the recycling industry since good quality recycling depends on the successful sorting of various materials. In textiles, the most commonly used fiber material types are wool, cotton, and polyester. When recycling fabrics, it is critical to identify and sort various fiber types quickly and correctly. The standard method of determining fabric fiber material type is the burn test followed by a microscopic examination. This traditional method is destructive, tedious, and slow since it involves cutting, burning, and examining the yarn of the fabric. We demonstrate that the identification procedure can be done nondestructively using optical coherence tomography (OCT) and deep learning. The OCT image scans of fabrics that are composed of different fiber material types such as wool, cotton, and polyester are used to train a deep neural network. We present the results of the created deep learning models’ capability to classify fabric fiber material types. We conclude that fiber material types can be identified nondestructively with high precision and recall by OCT imaging and deep learning. Because classification of material type can be performed by OCT and deep learning, this novel technique can be employed in recycling plants in sorting wool, cotton, and polyester fabrics automatically.