Development of a patients’ satisfaction analysis system using machine learning and lexicon-based methods

Abstract

Background Patients’ rights are integral to medical ethics. This study aimed to perform sentiment analysis and opinion mining on patients’ messages by a combination of lexicon-based and machine learning methods to identify positive or negative comments and to determine the different ward and staff names mentioned in patients’ messages. Methods The level of satisfaction and observance of the rights of 250 service recipients of the hospital was evaluated through the related checklists by the evaluator. In total, 822 Persian messages, composed of 540 negative and 282 positive comments, were collected and labeled by the evaluator. Pre-processing was performed on the messages and followed by 2 feature vectors which were extracted from the messages, including the term frequency–inverse document frequency (TFIDF) vector and a combination of the multifeature (MF) (a lexicon-based method) and TFIDF (MF + TFIDF) vectors. Six feature selectors and 5 classifiers were used in this study. For the evaluations, 5-fold cross-validation with different metrics including area under the receiver operating characteristic curve (AUC), accuracy (ACC), F1 score, sensitivity (SEN), specificity (SPE) and Precision-Recall Curves (PRC) were reported. Message tag detection, which featured different hospital wards and identified staff names mentioned in the study patients’ messages, was implemented by the lexicon-based method. Results The best classifier was Multinomial Naïve Bayes in combination with MF + TFIDF feature vector and SelectFromModel (SFM) feature selection (ACC = 0.89 ± 0.03, AUC = 0.87 ± 0.03, F1 = 0.92 ± 0.03, SEN = 0.93 ± 0.04, and SPE = 0.82 ± 0.02, PRC-AUC = 0.97). Two methods of assessment by the evaluator and artificial intelligence as well as survey systems were compared. Conclusion Our results demonstrated that the lexicon-based method, in combination with machine learning classifiers, could extract sentiments in patients’ comments and classify them into positive and negative categories. We also developed an online survey system to analyze patients’ satisfaction in different wards and to remove conventional assessments by the evaluator.

Use of sentiment analysis for capturing hospitalized cancer patients' experience from free-text comments in the Persian language

“…As mentioned in Table 7 , the existing SA methods include ML, deep learning, and lexicon-based methods [ 34 ]. In this research, a lexicon-based method was used.…”

“…There were few studies on SA in the Persian language [ 53 – 55 ] and so far, only one research [ 34 ] has been done in the field of healthcare. Our study is the first SA study conducted in the field of cancer in Iran.…”

“…Our study is the first SA study conducted in the field of cancer in Iran. Compared to research [ 34 ] we also used a lexicon-based method but did not use ML methods. No topic modeling methods were used in their study.…”

“…In both studies, the basis of the corpus was not the social network, but the information collected from the hospital's criticism and complaints form. In the corpus of the present study, similar to the study [ 34 ] patients' opinions about the care and non-care services of the hospital were analyzed. As a difference in our research, the life expectancy of cancer patients was also interpreted.…”

“…The first study in the field of healthcare using SA and machine learning in the Persian language was done by Khaleghparast et al [ 34 ] to identify various hospital wards and staff members from patients' remarks and to categorize messages into good and negative categories.…”