Analysis of Online Reviews of Orthopaedic Surgeons and Orthopaedic Practices Using Natural Language Processing

Abstract

Background: There is growing interest in measuring and improving patient experience. Machine learning–based natural language processing techniques may help identify instructive themes in online comments written by patients about their healthcare provider. Separating individual surgeon and orthopaedic office reviews, we analyzed themes that are discussed based on the rating category, the association with review length, the number of people posting more than one review for a surgeon or office, the mean number of reviews per rating category, and the difference in review tones. Methods: On Yelp.com, we collected 11,614 free-text reviews—together with a one- to five-star rating—of orthopaedic surgeons. Using natural language processing, we identified the most frequently occurring word combinations among rating categories. Themes were derived by categorizing word combinations. Dominant tones (emotional and language styles) were assessed by the IBM Watson Tone Analyzer. We calculated chi-square tests for linear trend and Spearman's rank correlation coefficients to assess differences among rating category. Results: For individual surgeons and orthopaedic offices, themes such as logistics, care and compassion, trust, recommendation, and customer service varied among rating categories. More positive reviews are shorter for individual surgeons and orthopaedic offices, while rating category was comparable among people posting more than one review for both groups. Tones of joy and confidence were associated with higher ratings. Sadness and tentative tones were associated with lower ratings. Discussion: For individual orthopaedic surgeons and orthopaedic offices, patient experience may be influenced mostly by the patient-clinician relationship. Training in more effective communication strategies may help improve self-reported patient experience.

Artificial intelligence in orthopaedics: false hope or not? A narrative review along the line of Gartner’s hype cycle

“…Patient experience reflects quality of care from the patient’s perspective, hence these are important data that can teach us about what creates an (un)satisfying experience. 35 …”

Emotional tones in scientific writing: comparison of commercially funded studies and non-commercially funded orthopedic studies

“… 2020 , Langerhuizen et al. 2020 ). 3rd, we included only studies in high-impact orthopedic journals.…”

“…2nd, the IBM tone analyzer was trained on a large Twitter customer-support dataset. Although previously used in the context of medical studies, the reliability of the analyzer for medical journals is untested (Ottenhoff et al 2018, Rajesh et al 2018, Bakker et al 2019, Karacic et al 2019, Black et al 2020, Langerhuizen et al 2020. 3rd, we included only studies in high-impact orthopedic journals.…”

Natural Language Processing and Its Use in Orthopaedic Research

“…They concluded that care and compassion are vital elements when it comes to patient satisfaction. Patient-clinician relationships and patient experiences seem to be related to communication and a lack of perceived empathy by the surgeon [ 16 •]. NLP and sentiment analysis were used to analyze free-text comments compared with a national inpatient survey by Greaves et al Over 6000 free-text comments posted online during the same period as the survey were analyzed.…”