A weakly supervised tooth‐mark and crack detection method in tongue image

Hui, Weng; Li, Lei; Lei, Huangwei; Luo, Zhiming; Li, Candong; Li, Shaozi

doi:10.1002/cpe.6262

Cited by 12 publications

(10 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When the results of accuracy in the validation do not increase, the training is stopped. The loss function for Faster R-CNN sums the classification loss and regression loss, as defined in the following equation [ 24 ]:

where N cls and N reg are the number of anchors in minibatch and number of anchor locations, λ and i mean the selected anchor box index and the balancing parameter; p i and p i ∗ represent the predicted probability and the ground truth of tongue feature; t i and t i ∗ represent the predicted bounding box and actual tongue feature label box. The accuracy results and the loss changes in the training are depicted in Figure 4(b) .…”

Section: Methodsmentioning

confidence: 99%

“…When the results of accuracy in the validation do not increase, the training is stopped. e loss function for Faster R-CNN sums the classification loss and regression loss, as defined in the following equation [24]:…”

Section: Model Training Validation and Testingmentioning

confidence: 99%

“…Xu et al [ 22 ] have proposed a CNN model combining a u-shaped net (U-NET) and Discriminative Filter Learning (DFL) for classification and recognition of different types of tongue coating, achieving an F1-Score of 93%. The research on the recognition and classification of the tooth-marked tongue [ 23 ] and cracked tongue [ 24 ] has significantly improved the accuracy of tongue image identification.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Learning Multi-label Tongue Image Analysis and Its Application in a Population Undergoing Routine Medical Checkup

Jiang

et al. 2022

Evidence-Based Complementary and Alternative Medicine

View full text Add to dashboard Cite

Background. Research on intelligent tongue diagnosis is a main direction in the modernization of tongue diagnosis technology. Identification of tongue shape and texture features is a difficult task for tongue diagnosis in traditional Chinese medicine (TCM). This study aimed to explore the application of deep learning techniques in tongue image analyses. Methods. A total of 8676 tongue images were annotated by clinical experts, into seven categories, including the fissured tongue, tooth-marked tongue, stasis tongue, spotted tongue, greasy coating, peeled coating, and rotten coating. Based on the labeled tongue images, the deep learning model faster region-based convolutional neural networks (Faster R-CNN) was utilized to classify tongue images. Four performance indices, i.e., accuracy, recall, precision, and F1-score, were selected to evaluate the model. Also, we applied it to analyze tongue image features of 3601 medical checkup participants in order to explore gender and age factors and the correlations among tongue features in diseases through complex networks. Results. The average accuracy, recall, precision, and F1-score of our model achieved 90.67%, 91.25%, 99.28%, and 95.00%, respectively. Over the tongue images from the medical checkup population, the model Faster R-CNN detected 41.49% fissured tongue images, 37.16% tooth-marked tongue images, 29.66% greasy coating images, 18.66% spotted tongue images, 9.97% stasis tongue images, 3.97% peeled coating images, and 1.22% rotten coating images. There were significant differences in the incidence of the fissured tongue, tooth-marked tongue, spotted tongue, and greasy coating among age and gender. Complex networks revealed that fissured tongue and tooth-marked were closely related to hypertension, dyslipidemia, overweight and nonalcoholic fatty liver disease (NAFLD), and a greasy coating tongue was associated with hypertension and overweight. Conclusion. The model Faster R-CNN shows good performance in the tongue image classification. And we have preliminarily revealed the relationship between tongue features and gender, age, and metabolic diseases in a medical checkup population.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Model Training Validation and Testingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning Multi-label Tongue Image Analysis and Its Application in a Population Undergoing Routine Medical Checkup

Jiang

et al. 2022

Evidence-Based Complementary and Alternative Medicine

View full text Add to dashboard Cite

show abstract

“…Li et al 7 proposed a new method using statistic feature extracted by wide line, such as Max-distance, to train a binary SVM as a classifier for cracked tongue. And now more deep learning methods are used for crack defection based on classification, pixel segmentation and object detection [8][9][10] 11 proposed to train the tooth-mark and crack detection model by using tongue images annotated boundingbox. It was a weakly supervised method that added several classification branches to recognize the tooth-marked tongue and cracked tongue according to the YOLO object detection model.…”

Section: Tongue Crack Recognition Using Segmentation Based Deep Learningmentioning

confidence: 99%

Tongue crack recognition using segmentation based deep learning

Yan

Cai

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Tongue cracks refer to fissures with different depth and shapes on the tongue’s surface, which can characterize the pathological characteristics of spleen and stomach. Tongue cracks are of great significance to the objective study of tongue diagnosis. However, tongue cracks are small and complex, existing methods are difficult to extract them effectively. In order to achieve more accurate extraction and identification of tongue crack, this paper proposes to apply a deep learning network based on image segmentation (Segmentation-Based Deep-Learning, SBDL) to extract and identify tongue crack. In addition, we have studied the quantitative description of tongue crack features. Firstly, the pre-processed tongue crack samples were amplified by using adding salt and pepper noise, changing the contrast and horizontal mirroring; secondly, the annotation tool Crack-Tongue was used to label tongue crack; thirdly, the tongue crack extraction model was trained by using SBDL; fourthly, the cracks on the tongue surface were detected and located by the segmentation network, and then the output and features of the segmentation network were put into the decision network for the classification of crack tongue images; finally, the tongue crack segmentation and identification results were quantitatively evaluated. The experimental results showed that the tongue crack extraction and recognition results based on SBDL were better than Mask Region-based Convolutional Neural Network (Mask R-CNN), DeeplabV3+, U-Net, UNet++ and Semantic Segmentation with Adversarial Learning (SegAN). This method effectively solved the inaccurate tongue crack extraction caused by the tongue crack’s color being close to the surrounding tongue coating’s color. This method can achieve better tongue crack extraction and recognition results on a small tongue crack data set and provides a new idea for tongue crack recognition, which is of practical value for tongue diagnosis objectification.

show abstract

“…However, it requires a lot of image annotation including tongue landmark annotation and tongue region annotation, which is a huge burden and tedious work for clinic. Weng et al (2021) proposed a weakly supervised tooth-mark detection method using the YOLO object detection model. However, it requires fully bounding-box level annotation of tooth marks in addition to coarse image-level annotation of tooth-marked tongue images.…”

Section: Introductionmentioning

confidence: 99%

Weakly Supervised Deep Learning for Tooth-Marked Tongue Recognition

Zhou

Wang

et al. 2022

Front. Physiol.

View full text Add to dashboard Cite

The recognition of tooth-marked tongues has important value for clinical diagnosis of traditional Chinese medicine. Tooth-marked tongue is often related to spleen deficiency, cold dampness, sputum, effusion, and blood stasis. The clinical manifestations of patients with tooth-marked tongue include loss of appetite, borborygmus, gastric distention, and loose stool. Traditional clinical tooth-marked tongue recognition is conducted subjectively based on the doctor’s visual observation, and its performance is affected by the doctor’s subjectivity, experience, and environmental lighting changes. In addition, the tooth marks typically have various shapes and colors on the tongue, which make it very challenging for doctors to identify tooth marks. The existing methods based on deep learning have made great progress for tooth-marked tongue recognition, but there are still shortcomings such as requiring a large amount of manual labeling of tooth marks, inability to detect and locate the tooth marks, and not conducive to clinical diagnosis and interpretation. In this study, we propose an end-to-end deep neural network for tooth-marked tongue recognition based on weakly supervised learning. Note that the deep neural network only requires image-level annotations of tooth-marked or non-tooth marked tongues. In this method, a deep neural network is trained to classify tooth-marked tongues with the image-level annotations. Then, a weakly supervised tooth-mark detection network (WSTDN) as an architecture variant of the pre-trained deep neural network is proposed for the tooth-marked region detection. Finally, the WSTDN is re-trained and fine-tuned using only the image-level annotations to simultaneously realize the classification of the tooth-marked tongue and the positioning of the tooth-marked region. Experimental results of clinical tongue images demonstrate the superiority of the proposed method compared with previously reported deep learning methods for tooth-marked tongue recognition. The proposed tooth-marked tongue recognition model may provide important syndrome diagnosis and efficacy evaluation methods, and contribute to the understanding of ethnopharmacological mechanisms.

show abstract

A weakly supervised tooth‐mark and crack detection method in tongue image

Cited by 12 publications

References 37 publications

Deep Learning Multi-label Tongue Image Analysis and Its Application in a Population Undergoing Routine Medical Checkup

Deep Learning Multi-label Tongue Image Analysis and Its Application in a Population Undergoing Routine Medical Checkup

Tongue crack recognition using segmentation based deep learning

Weakly Supervised Deep Learning for Tooth-Marked Tongue Recognition

Contact Info

Product

Resources

About