Constructing tongue coating recognition model using deep transfer learning to assist syndrome diagnosis and its potential in noninvasive ethnopharmacological evaluation

Wang, Xu; Lou, Yanni; Liu, Jingwei; Huo, Shirui; Pang, Xiaohan; Wang, Weilu; Wu, Chaoyong; Chen, Yufeng; Chen, Aiping; Bi, Fukun; Xing, Weiying; Deng, Qi; Jia, Liqun; Chen, Jianxin

doi:10.1016/j.jep.2021.114905

Cited by 21 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Intelligent tongue diagnosis is one part of the important content in clinical TCM diagnosis. e researchers have applied the tongue image features extracted by deep learning to diabetes mellitus [4,42,43], NAFLD [44], lung cancerassisted diagnosis [45], and TCM constitution recognition [46][47][48] with good performance of disease classification [13,49]. Professor Yang Junlin's team [50] has applied the AI screening system for scoliosis developed and established by Faster R-CNN, and quantified the severity of scoliosis, with the accuracy reaching the average level of human experts.…”

Section: Discussionmentioning

confidence: 99%

“…Intelligent diagnosis based on images is a main direction of modernization of tongue diagnosis technology [13]. As the current mainstream technology, a convolutional neural network (CNN) has a powerful capability of feature extraction and representation [14,15], which greatly improves the accuracy and efficiency of tongue image segmentation, and classification [16][17][18][19][20].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning Multi-label Tongue Image Analysis and Its Application in a Population Undergoing Routine Medical Checkup

Jiang

et al. 2022

Evidence-Based Complementary and Alternative Medicine

View full text Add to dashboard Cite

Background. Research on intelligent tongue diagnosis is a main direction in the modernization of tongue diagnosis technology. Identification of tongue shape and texture features is a difficult task for tongue diagnosis in traditional Chinese medicine (TCM). This study aimed to explore the application of deep learning techniques in tongue image analyses. Methods. A total of 8676 tongue images were annotated by clinical experts, into seven categories, including the fissured tongue, tooth-marked tongue, stasis tongue, spotted tongue, greasy coating, peeled coating, and rotten coating. Based on the labeled tongue images, the deep learning model faster region-based convolutional neural networks (Faster R-CNN) was utilized to classify tongue images. Four performance indices, i.e., accuracy, recall, precision, and F1-score, were selected to evaluate the model. Also, we applied it to analyze tongue image features of 3601 medical checkup participants in order to explore gender and age factors and the correlations among tongue features in diseases through complex networks. Results. The average accuracy, recall, precision, and F1-score of our model achieved 90.67%, 91.25%, 99.28%, and 95.00%, respectively. Over the tongue images from the medical checkup population, the model Faster R-CNN detected 41.49% fissured tongue images, 37.16% tooth-marked tongue images, 29.66% greasy coating images, 18.66% spotted tongue images, 9.97% stasis tongue images, 3.97% peeled coating images, and 1.22% rotten coating images. There were significant differences in the incidence of the fissured tongue, tooth-marked tongue, spotted tongue, and greasy coating among age and gender. Complex networks revealed that fissured tongue and tooth-marked were closely related to hypertension, dyslipidemia, overweight and nonalcoholic fatty liver disease (NAFLD), and a greasy coating tongue was associated with hypertension and overweight. Conclusion. The model Faster R-CNN shows good performance in the tongue image classification. And we have preliminarily revealed the relationship between tongue features and gender, age, and metabolic diseases in a medical checkup population.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep Learning Multi-label Tongue Image Analysis and Its Application in a Population Undergoing Routine Medical Checkup

Jiang

et al. 2022

Evidence-Based Complementary and Alternative Medicine

View full text Add to dashboard Cite

show abstract

“…Our model may provide an important research paradigm for distinguishing tongue features, diagnosing syndromes of traditional Chinese medicine, tracking disease progression, and evaluating intervention effects, showing its unique potential in clinical applications. Potentially, the proposed method can also be used to evaluate the efficacy of the drug by detecting the tooth marks of the tongue for noninvasive ethnopharmacological evaluation ( Wang et al, 2022 ). The pathological cause of tooth-marked tongue is the change of microcirculation of the tongue due to the compression of the tongue by the teeth.…”

Section: Discussionmentioning

confidence: 99%

Weakly Supervised Deep Learning for Tooth-Marked Tongue Recognition

Zhou

Wang

et al. 2022

Front. Physiol.

Self Cite

View full text Add to dashboard Cite

The recognition of tooth-marked tongues has important value for clinical diagnosis of traditional Chinese medicine. Tooth-marked tongue is often related to spleen deficiency, cold dampness, sputum, effusion, and blood stasis. The clinical manifestations of patients with tooth-marked tongue include loss of appetite, borborygmus, gastric distention, and loose stool. Traditional clinical tooth-marked tongue recognition is conducted subjectively based on the doctor’s visual observation, and its performance is affected by the doctor’s subjectivity, experience, and environmental lighting changes. In addition, the tooth marks typically have various shapes and colors on the tongue, which make it very challenging for doctors to identify tooth marks. The existing methods based on deep learning have made great progress for tooth-marked tongue recognition, but there are still shortcomings such as requiring a large amount of manual labeling of tooth marks, inability to detect and locate the tooth marks, and not conducive to clinical diagnosis and interpretation. In this study, we propose an end-to-end deep neural network for tooth-marked tongue recognition based on weakly supervised learning. Note that the deep neural network only requires image-level annotations of tooth-marked or non-tooth marked tongues. In this method, a deep neural network is trained to classify tooth-marked tongues with the image-level annotations. Then, a weakly supervised tooth-mark detection network (WSTDN) as an architecture variant of the pre-trained deep neural network is proposed for the tooth-marked region detection. Finally, the WSTDN is re-trained and fine-tuned using only the image-level annotations to simultaneously realize the classification of the tooth-marked tongue and the positioning of the tooth-marked region. Experimental results of clinical tongue images demonstrate the superiority of the proposed method compared with previously reported deep learning methods for tooth-marked tongue recognition. The proposed tooth-marked tongue recognition model may provide important syndrome diagnosis and efficacy evaluation methods, and contribute to the understanding of ethnopharmacological mechanisms.

show abstract

“…The different characteristics of the tongue (such as color, shape, etc.) were significantly related to the health condition and different diseases [6][7][8], such as gastritis [9,10], cancer [11][12][13], Corona Virus Disease-19 (COVID-19) [14,15], diabetes [16][17][18]. Therefore, tongue diagnosis can be used for health monitoring and the prediction of diseases.…”

Section: Introductionmentioning

confidence: 99%

Development of attention-based robust deep learning model for tongue diagnosis by smartphone

Gao

Hou

et al. 2023

Preprint

View full text Add to dashboard Cite

Compared with tongue diagnosis using tongue image analyzers, tongue diagnosis by smartphone has great advantages in convenience and cost for universal health monitoring, but its accuracy is affected by the shooting conditions of smartphones. Developing deep learning models with high accuracy and robustness to changes in the shooting environment for tongue diagnosis by smartphone and determining the influence of environmental changes on accuracy are necessary. In our study, a dataset of 9003 images was constructed after image pre-processing and labeling. Next, we developed an attention-based deep learning model (Deep Tongue) for 8 subtasks of tongue diagnosis, including the spotted tongue, teeth-marked tongue, and fissure tongue et al, which the average AUC of was 0.90, higher than the baseline model (ResNet50) by 0.10. Finally, we analyzed the objective reasons, the brightness of the environment and the hue of images, affecting the accuracy of tongue diagnosis by smartphone through a consistency experiment of direct subject inspection and tongue image inspection. Finally, we determined the influence of environmental changes on accuracy to quantify the robustness of the Deep Tongue model through simulation experiments. Overall, the Deep Tongue model achieved a higher and more stable classification accuracy of seven tongue diagnosis tasks in the complex shooting environment of the smartphone, and the classification of tongue coating (yellow/white) was found to be sensitive to the hue of the images and therefore unreliable without stricter shooting conditions and color correction.

show abstract

Constructing tongue coating recognition model using deep transfer learning to assist syndrome diagnosis and its potential in noninvasive ethnopharmacological evaluation

Cited by 21 publications

References 18 publications

Deep Learning Multi-label Tongue Image Analysis and Its Application in a Population Undergoing Routine Medical Checkup

Deep Learning Multi-label Tongue Image Analysis and Its Application in a Population Undergoing Routine Medical Checkup

Weakly Supervised Deep Learning for Tooth-Marked Tongue Recognition

Development of attention-based robust deep learning model for tongue diagnosis by smartphone

Contact Info

Product

Resources

About