Effects of Questionnaire-Based Diagnosis and Training on Inter-Rater Reliability Among Practitioners of Traditional Chinese Medicine

Abstract

Objective: To investigate whether a training process that focused on a questionnaire-based diagnosis in Traditional Chinese Medicine (TCM), and developing diagnostic consensus, would improve the agreement of TCM diagnoses among 10 TCM practitioners evaluating patients with temporomandibular joint disorder (TMJD). Design and setting: Evaluation of a diagnostic training program at the Department of Family and Community Medicine, University of Arizona, Tucson Arizona, and the Oregon College of Oriental Medicine, Portland, Oregon. Subjects: Screened participants for a study of TCM for TMJD. Practitioners: Ten (10) licensed acupuncturists with a minimum of 5 years licensure and education in Chinese herbs. Methods: A training session using a questionnaire-based diagnostic form was conducted, followed by waves of diagnostic sessions. Between sessions, practitioners discussed the results of the previous round of participants with a focus on reducing variability in primary diagnosis and severity rating of each diagnosis: 3 waves of 5 patients were assessed by 4 practitioner pairs for a total of 120 diagnoses. At 18 months, practitioners completed a recalibration exercise with a similar format with a total of 32 diagnoses. These diagnoses were then examined with respect to the rate of agreement among the 10 practitioners using inter-rater correlations and kappas. Results: The inter-rater correlation with respect to the TCM diagnoses among the 10 practitioners increased from 0.112 to 0.618 with training. Statistically significant improvements were found between the baseline and 18 month exercises ( p < 0.01). Conclusions: Inter-rater reliability of TCM diagnosis may be improved through a training process and a questionnaire-based diagnosis process. The improvements varied by diagnosis, with the greatest congruence among primary and more severe diagnoses. Future TCM studies should consider including calibration training to improve the validity of results.

TCM tongue diagnosis index of early-stage breast cancer

“…Previous studies have been conducted on the issue regarding consistency of TCM diagnosis [9][10][11][12][13][14][15][16][17][18][19] as well as herbal prescription or treatment, 13,14,16 indicating that inter-and intra-observer agreements are low. The inconsistency of subjective diagnosis and treatment can be improved by the development of validated instruments, such as standard questionnaire for inquiries 5 and manual for guiding treatments.…”

Validity of a Diagnostic Scale for Acupuncture: Application of the Item Response Theory to the Five Viscera Score

“…In acupuncture therapy, diagnosis, acupoints, and stimulation methods for a patient or patients with the same illness often vary between TCM practitioners [6–8]. Scientists are conducting intervention studies for specific diseases to evaluate the effectiveness of acupuncture therapy.…”

A Review of Studies of Tongue Deviations in Korean Stroke Patients

“…Zhang et al [32] analyzed the effect of training on improved agreement in TCM diagnosis among its practitioners based on a sample of 42 patients with rheumatoid arthritis (RA). Mist et al [33], using interrater correlations and kappa values, assessed whether a training process that focused on a questionnaire-based diagnosis would improve agreement in traditional Chinese medicine TCM diagnosis. Finally, Kim et al [34] examined the reliability of TCM tongue inspection by evaluating inter-and intrapractitioner agreement levels for specific tongue characteristics.…”