User-Initiated Symptom Assessment With an Electronic Symptom Checker: Protocol for a Mixed Methods Validation Study

Liu, Ville Dan Mikael; Kaila, Minna; Koskela, Tuomas

doi:10.2196/41423

Triage Accuracy and the Safety of User-Initiated Symptom Assessment With an Electronic Symptom Checker in a Real-Life Setting: Instrument Validation Study

Liu,

Kaila,

Koskela

2024

JMIR Hum Factors

0

View full text Add to dashboard Cite

Background: Previous studies have evaluated the accuracy of the diagnostics of electronic symptom checkers (ESCs) and triage using clinical case vignettes. National Omaolo Digital services (Omaolo) in Finland consist of an ESC for various symptoms. Omaolo is a medical device (based on Duodecim Clinical Decision Support EBMEDS) with a CE marking (risk class: IIa).Objective: This study investigates how well triage performed by the ESC matches the triage of nurses among the chief symptom assessments available in Omaolo (anal region symptoms, cough, diarrhea, discharge from the eye or watery or reddish eye, headache, heartburn, knee symptom or injury, lower back pain or injury, oral health, painful or blocked ear, respiratory tract infection, sexually transmitted disease, shoulder pain or stiffness or injury, sore throat or throat symptom, urinary tract infection). In addition, this study assesses the accuracy, specificity, sensitivity and safety of the Omaolo ESC.Methods: This is a clinical validation study in a real-life setting. It is a multicenter study in primary health care (PHC) centers across Finland. The included units were of the walk-in model of primary care, where no prior phone call or online contact was required. Upon arriving at the PHC center, users (patients) filled out the ESC questions, resulting in a triage recommendation, and after that a nurse assessed their triage. Findings concerning 877 patients were analyzed. The triage recommendations by the ESC were matched with the triage given by the triage nurse. Results:The mean of the exact match for all symptom assessments were in 471out of 877, 53.7% (CI95 49.2%, 55.9%). The mean of the exact match or overly conservative but suitable (ESC's assessment was one triage level higher than the nurse's triage) for all symptom assessments were in 584 out of 877, 66.6% (CI95 63.4%, 69.7%). Safe assessments by the ESC accounted for 856 of 877, 97.6% (CI95 95.6%, 98.0%) of all assessments made. Concerning acute cases in which the nurse evaluated that a user needed to be treated urgently the ESC's exactly matched accuracy were in 244 out of 344, 70.9%, (CI95 65.8%, 75.7%). Sensitivity for the Omaolo ESC was 62.6% and specificity of 69.2%. Critical assessments were identified and further analyzed. In these 21 assessments, no indication was found that patient safety had been compromised. Conclusions:This study was the first to assess the accuracy, specificity, sensitivity and safety of the Omaolo ESC in the Finnish PHC context. The findings indicate that the Omaolo ESC is safe to use when compared to the triage assessment of a nurse. Omaolo ESC has the potential to direct the patient user to the right place at the right time in terms of triage.

show abstract

Triage Accuracy and the Safety of User-initiated Symptom Assessment with an Electronic Symptom Checker in a Real-life Setting: Comparison Study (Preprint)

Liu,

Koskela,

Kaila

2023

Preprint

0

View full text Add to dashboard Cite

BACKGROUND Previous studies have evaluated the accuracy of the diagnostics of electronic symptom checkers (ESCs) and triage using clinical case vignettes. National Omaolo Digital services (Omaolo) in Finland consist of an ESC for various symptoms. Omaolo is a medical device (based on Duodecim Clinical Decision Support EBMEDS) with a CE marking (risk class: IIa). OBJECTIVE This study investigates how well triage performed by the ESC matches the triage of nurses among the chief symptom assessments available in Omaolo (anal region symptoms, cough, diarrhea, discharge from the eye or watery or reddish eye, headache, heartburn, knee symptom or injury, lower back pain or injury, oral health, painful or blocked ear, respiratory tract infection, sexually transmitted disease, shoulder pain or stiffness or injury, sore throat or throat symptom, urinary tract infection). In addition, this study assesses the accuracy, specificity, sensitivity and safety of the Omaolo ESC. METHODS This is a clinical validation study in a real-life setting. It is a multicenter study in primary health care (PHC) centers across Finland. The included units were of the walk-in model of primary care, where no prior phone call or online contact was required. Upon arriving at the PHC center, users (patients) filled out the ESC questions, resulting in a triage recommendation, and after that a nurse assessed their triage. Findings concerning 877 patients were analyzed. The triage recommendations by the ESC were matched with the triage given by the triage nurse. RESULTS The mean of the exact match for all symptom assessments were in 471out of 877, 53.7% (CI95 49.2%, 55.9%). The mean of the exact match or overly conservative but suitable (ESC’s assessment was one triage level higher than the nurse’s triage) for all symptom assessments were in 584 out of 877, 66.6% (CI95 63.4%, 69.7%). Safe assessments by the ESC accounted for 856 of 877, 97.6% (CI95 95.6%, 98.0%) of all assessments made. Concerning acute cases in which the nurse evaluated that a user needed to be treated urgently the ESC’s exactly matched accuracy were in 244 out of 344, 70.9%, (CI95 65.8%, 75.7%). Sensitivity for the Omaolo ESC was 62.6% and specificity of 69.2%. Critical assessments were identified and further analyzed. In these 21 assessments, no indication was found that patient safety had been compromised. CONCLUSIONS This study was the first to assess the accuracy, specificity, sensitivity and safety of the Omaolo ESC in the Finnish PHC context. The findings indicate that the Omaolo ESC is safe to use when compared to the triage assessment of a nurse. Omaolo ESC has the potential to direct the patient user to the right place at the right time in terms of triage.

show abstract

Triage Accuracy and the Safety of User-Initiated Symptom Assessment With an Electronic Symptom Checker in a Real-Life Setting: Instrument Validation Study (Preprint)

Liu,

Kaila,

Koskela

2023

Preprint

0

View full text Add to dashboard Cite

BACKGROUND Previous studies have evaluated the accuracy of the diagnostics of electronic symptom checkers (ESCs) and triage using clinical case vignettes. National Omaolo Digital services (Omaolo) in Finland consist of an ESC for various symptoms. Omaolo is a medical device (based on Duodecim Clinical Decision Support EBMEDS) with a CE marking (risk class: IIa). OBJECTIVE This study investigates how well triage performed by the ESC matches the triage of nurses among the chief symptom assessments available in Omaolo (anal region symptoms, cough, diarrhea, discharge from the eye or watery or reddish eye, headache, heartburn, knee symptom or injury, lower back pain or injury, oral health, painful or blocked ear, respiratory tract infection, sexually transmitted disease, shoulder pain or stiffness or injury, sore throat or throat symptom, urinary tract infection). In addition, this study assesses the accuracy, specificity, sensitivity and safety of the Omaolo ESC. METHODS This is a clinical validation study in a real-life setting. It is a multicenter study in primary health care (PHC) centers across Finland. The included units were of the walk-in model of primary care, where no prior phone call or online contact was required. Upon arriving at the PHC center, users (patients) filled out the ESC questions, resulting in a triage recommendation, and after that a nurse assessed their triage. Findings concerning 877 patients were analyzed. The triage recommendations by the ESC were matched with the triage given by the triage nurse. RESULTS The mean of the exact match for all symptom assessments were in 471out of 877, 53.7% (CI95 49.2%, 55.9%). The mean of the exact match or overly conservative but suitable (ESC’s assessment was one triage level higher than the nurse’s triage) for all symptom assessments were in 584 out of 877, 66.6% (CI95 63.4%, 69.7%). Safe assessments by the ESC accounted for 856 of 877, 97.6% (CI95 95.6%, 98.0%) of all assessments made. Concerning acute cases in which the nurse evaluated that a user needed to be treated urgently the ESC’s exactly matched accuracy were in 244 out of 344, 70.9%, (CI95 65.8%, 75.7%). Sensitivity for the Omaolo ESC was 62.6% and specificity of 69.2%. Critical assessments were identified and further analyzed. In these 21 assessments, no indication was found that patient safety had been compromised. CONCLUSIONS This study was the first to assess the accuracy, specificity, sensitivity and safety of the Omaolo ESC in the Finnish PHC context. The findings indicate that the Omaolo ESC is safe to use when compared to the triage assessment of a nurse. Omaolo ESC has the potential to direct the patient user to the right place at the right time in terms of triage.

show abstract

User-Initiated Symptom Assessment With an Electronic Symptom Checker: Protocol for a Mixed Methods Validation Study

Cited by 3 publications

References 64 publications

Triage Accuracy and the Safety of User-Initiated Symptom Assessment With an Electronic Symptom Checker in a Real-Life Setting: Instrument Validation Study

Triage Accuracy and the Safety of User-Initiated Symptom Assessment With an Electronic Symptom Checker in a Real-Life Setting: Instrument Validation Study

Triage Accuracy and the Safety of User-initiated Symptom Assessment with an Electronic Symptom Checker in a Real-life Setting: Comparison Study (Preprint)

Triage Accuracy and the Safety of User-Initiated Symptom Assessment With an Electronic Symptom Checker in a Real-Life Setting: Instrument Validation Study (Preprint)

Contact Info

Product

Resources

About