Content and Usability Validation of an Intelligent Virtual Conversation Assistant Used for Virtual Triage During the COVID-19 Pandemic in Brazil

Abstract

This study aimed to describe the development process, content validation, and usability of a COVID-19 screening system incorporated into a chatbot-type intelligent virtual assistant (CoronaBot). This is a methodological research carried out in three phases. The first corresponded to the development of the flowchart and content of the virtual assistant, the second phase consisted of the implementation of the content in chatbot, and the third phase consisted of content validation. Data analysis was performed by agreement rate, content validity index, and kappa statistical test. Finally, in the third phase, the chatbot's usability was analyzed using the System Usability Scale, by 10 users. The CoronaBot content presented domains with agreement rate above 87.5%, and its items referring to symptomatological scores and interface screens had values of content validity index with a mean of 0.96, kappa test with values from 0.70 to 0.76, and interspecialist agreement of 1.00, demonstrating excellence of prototype content. The global usability score was 80.1. The script developed and incorporated into the chatbot prototype achieved a satisfactory level of content validity. The usability of the chatbot was considered good, adding to the credibility of the device.

The “Flapbot”: A Global Perspective on the Validity and Usability of a Flap Monitoring Chatbot

“…Furthermore, another study introduced “Coronabot,” an AI-based tool facilitating virtual coronavirus disease 2019 symptom assessment and provision of appropriate advice. 25 Comparable to the study evaluating the wound care chatbot, validity and usability were measured through S-CVI and SUS, scoring at 0.96 and 80.1, respectively. In comparison to these studies, the Flapbot achieved a S-CVI of 0.82 and a SUS score of 68, reflecting excellent validity but average usability.…”

“…The chatbot's content was ratified by expert opinion, earning an impressive S-CVI score of 0.97 and an SUS score of 80.1, indicating exceptional validity and good usability. Furthermore, another study introduced 'Coronabot', an AI-based tool facilitating virtual COVID-19 symptom assessment and provision of appropriate advice 25 . Comparable to the study evaluating the wound care chatbot, validity and usability were measured through S-CVI and SUS, scoring at 0.96 and 80.1, respectively.…”

Telephone Triage Services in Canada

“…• emergency department triage [91][92][93][94][95] • screening patients with suspected COVID-19 [96][97][98] • tele-dermatology case prioritization� 99…”