SERIES: eHealth in primary care. Part 5: A critical appraisal of five widely used eHealth applications for primary care – opportunities and challenges

Kasteleyn, Marise J.; Versluis, Anke; Peet, Petra G. van; Kirk, Ulrik Bak; Dalfsen, Jens van; Meijer, Eline; Honkoop, Persijn; Ho, Kendall; Chavannes, Niels H.; Talboom-Kamp, Esther

doi:10.1080/13814788.2021.1962845

Cited by 11 publications

(15 citation statements)

References 40 publications

(71 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Third, our intervention content was integrated into app-based interventions that are used in a blended care setting in primary and secondary care. There is a growing body of literature that recognizes that blended interventions are more likely to be used and effective than stand-alone interventions [ 36 , 75 - 77 , 79 - 81 ]. By combining aforementioned elements, we expect that our intervention could positively contribute to sustainable health behavior change, although this still needs to be researched.…”

Section: Discussionmentioning

confidence: 99%

A Digital Lifestyle Coach (E-Supporter 1.0) to Support People With Type 2 Diabetes: Participatory Development Study

Hietbrink¹,

Middelweerd²,

Empelen³

et al. 2023

JMIR Hum Factors

View full text Add to dashboard Cite

Background A healthy lifestyle, including regular physical activity and a healthy diet, is becoming increasingly important in the treatment of chronic diseases. eHealth interventions that incorporate behavior change techniques (BCTs) and dynamic tailoring strategies could effectively support a healthy lifestyle. E-Supporter 1.0 is an eCoach designed to support physical activity and a healthy diet in people with type 2 diabetes (T2D). Objective This paper aimed to describe the systematic development of E-Supporter 1.0. Methods Our systematic design process consisted of 3 phases. The definition phase included the selection of the target group and formulation of intervention objectives, and the identification of behavioral determinants based on which BCTs were selected to apply in the intervention. In the development phase, intervention content was developed by specifying tailoring variables, intervention options, and decision rules. In the last phase, E-Supporter 1.0 integrated in the Diameter app was evaluated using a usability test in 9 people with T2D to assess intervention usage and acceptability. Results The main intervention objectives were to stimulate light to moderate-vigorous physical activities or adherence to the Dutch dietary guidelines in people with T2D. The selection of behavioral determinants was informed by the health action process approach and theories explaining behavior maintenance. BCTs were included to address relevant behavioral determinants (eg, action control, self-efficacy, and coping planning). Development of the intervention resulted in 3 types of intervention options, consisting of motivational messages, behavioral feedback, and tailor-made supportive exercises. On the basis of IF-THEN rules, intervention options could be tailored to, among others, type of behavioral goal and (barriers to) goal achievement. Data on these variables could be collected using app data, activity tracker data, and daily ecological momentary assessments. Usability testing revealed that user experiences were predominantly positive, despite some problems in the fixed delivery of content. Conclusions The systematic development approach resulted in a theory-based and dynamically tailored eCoach. Future work should focus on expanding intervention content to other chronic diseases and lifestyle behaviors, enhancing the degree of tailoring and evaluating intervention effects on acceptability, use, and cost-effectiveness.

show abstract

Section: Discussionmentioning

confidence: 99%

A Digital Lifestyle Coach (E-Supporter 1.0) to Support People With Type 2 Diabetes: Participatory Development Study

Hietbrink¹,

Middelweerd²,

Empelen³

et al. 2023

JMIR Hum Factors

View full text Add to dashboard Cite

show abstract

“…Another strength is that Homelab was developed in cocreation with GPs. Cocreation in eHealth interventions is an important precondition for good adoption of eHealth [11]. Homelab was piloted and developed for general practices in the Netherlands.…”

Section: Limitations and Strengths Of This Studymentioning

confidence: 99%

“…Homelab was piloted and developed for general practices in the Netherlands. However, a service like Homelab can also be implemented in other European countries with comparable primary health care systems where the GP is the first gatekeeper-in particular, Nordic countries are relatively advanced in adopting eHealth [11].…”

Section: Limitations and Strengths Of This Studymentioning

confidence: 99%

“…For example, in the Netherlands, a noncommercial website was developed by GPs for citizens to obtain reliable health information [8], and a significant decrease in the consultations was noted after the website's launch compared to the total consultations before the launch [8,9]. Apps that support lifestyle change or the self-management of chronic diseases (eg, promoting physical activity, healthy diet, weight management) can also benefit GPs, as these apps can take over part of the GPs' coaching [10][11][12]. Consequently, GPs may have more time for other health care activities.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Usability of Homelab, a Digital Self-service at a Dutch General Practice, for Diagnostic Tests: Pilot Study With a Questionnaire

Schnoor¹,

Versluis²,

Chavannes³

et al. 2023

JMIR Form Res

Self Cite

View full text Add to dashboard Cite

Background eHealth potentially can make health care more accessible and efficient and help reduce the workload in primary health care. Homelab is an eHealth tool implemented in a general practice environment, and it offers relatively simple laboratory diagnostic tests without the referral of the general practitioner. After logging in this eHealth tool, patients select and order a diagnostic test based on their symptoms. The test results are presented online to the general practitioner and the patient. Objective This study aims to evaluate the use, usability, and user characteristics of Homelab. Further, it aims to evaluate whether Homelab replaces an appointment with the general practitioner. Methods Homelab has been implemented since May 2021 as a pilot in a Dutch general practice. The number of requests and the ordered diagnostic packages are monitored. After using Homelab, patients are invited to complete a short questionnaire. The questionnaire contains demographic questions and assesses usability using the System Usability Scale (10 items). In addition, questions about requesting an appointment with the general practitioner without Homelab are included. All data were anonymous. Results The questionnaire was filled by 74 individual patients. The mean age of the patients was 40.33 (SD 12.11) years, and half of them were females (39/74, 53%). The majority of the patients were highly educated (56/74, 76%) and employed (53/74, 72%). Approximately 81% (60/74) of the patients reported that they would use Homelab again in the future and 66% (49/74) reported that they would have gone to the general practitioner if they had not used Homelab. The usability of Homelab was perceived higher by the younger age group (mean 73.96, SD 14.74) than by the older age group (mean 61.59, SD 14.37). In total, 106 test packages were ordered over 1 year, and the most requested diagnostic package was “Am I still healthy? I want to do my annual health checkup.” Homelab was used the most during the months of the COVID-19 lockdown. Conclusions The use of Homelab, a digital self-service for ordering diagnostic tests, was monitored in this study, and its usability was perceived as above average. Our findings showed that patients are willing to use Homelab in the future and they would use it most of the time as a replacement for regular consultations. Homelab offers opportunities for more accessible and efficient health care for both the patient and the general practitioner.

show abstract

“…A recent study has showed that most patients (more than 76%) use checkers solely for self-diagnosis [39]. As such, if checkers are not meticulously engineered and rigorously evaluated on their diagnostic capabilities, they may put patients at risk [4, 21, 33]. To this end, this paper focuses on verifying the diagnostic accuracy of checkers due to serving as the underpinning of any aspired benefit.…”

Section: Introductionmentioning

confidence: 99%

Avey: An Accurate AI Algorithm for Self-Diagnosis

Hammoud

Douglas

Darmach

et al. 2022

Preprint

View full text Add to dashboard Cite

ObjectivesTo present our AI-based symptom checker, rigorously measure its accuracy, and compare it against existing popular symptom checkers and seasoned primary care physicians.DesignVignettes study.Setting400 gold-standard primary care vignettes.Intervention/ComparatorWe utilized 7 standard accuracy metrics for evaluating the performance of 6 symptom checkers. To this end, we developed and peer-reviewed 400 vignettes, each approved by at least 5 out of 7 independent and experienced general practitioners. To the best of our knowledge, this yielded the largest benchmark vignette suite in the field thus far. To establish a frame of reference and interpret the results of symptom checkers accordingly, we further directly compared the best-performing symptom checker against 3 primary care physicians with an average experience of 16.6 years.Primary Outcome MeasuresWe thoroughly studied the diagnostic accuracies of symptom checkers and physicians from 7 standard angles, including: (a) M1, M3, and M5 as measures of a symptom checker’s or a physician’s ability to return a vignette’s main diagnosis at the top, among the first 3 diseases, or among the first 5 diseases of their differential diagnosis, respectively (b) recall as a measure of the percentage of relevant diseases that are returned in a symptom checker’s or a physician’s differential diagnosis, (c) precision as a measure of the percentage of diseases in a symptom checker’s or a physician’s differential diagnosis that are relevant, (d) F1-measure as a trade-off measure between recall and precision, and (e) Normalized Discounted Cumulative Gain or NDCG as a measure of ranking quality of a symptom checker’s or a physician’s differential diagnosis.ResultsOur AI-based symptom checker, namely, Avey significantly outperformed 5 popular symptom checkers, namely, Ada, WebMD, K Health, Buoy, and Babylon by averages of 24.5%, 175.5%, 142.8%, 159.6%, 2968.1% using M1; 22.4%, 114.5%, 123.8%, 118.2%, 3392% using M3; 18.1%, 79.2%, 116.8%, 125%, 3114.2% using M5; 25.2%, 65.6%, 109.4%, 154%, 3545% using recall; 8.7%, 88.9%, 66.4%, 88.9%, 2084% using F1-measure; and 21.2%, 93.4%, 113.3%, 136.4%, 3091.6% using NDCG, respectively. Under precision, Ada outperformed Avey by an average of 0.9%, while Avey surpassed WebMD, K Health, Buoy, and Babylon by averages of 103.2%, 40.9%, 49.6%, and 1148.5%, respectively. To the contrary of symptom checkers, physicians outperformed Avey by averages of 37.1% and 1.2% using precision and F1-measure, while Avey exceeded them by averages of 10.2%, 20.4%, 23.4%, 56.4%, and 25.1% using M1, M3, M5, recall, and NDCG, respectively. To facilitate the reproducibility of our study and support future related studies, we made all our gold-standard vignettes publicly and freely available. Moreover, we posted online all the results of the symptoms checkers and physicians (i.e., 45 sets of experiments) to establish a standard of full transparency and enable verifying and cross validating our results.ConclusionsAvey tremendously outperformed the considered symptom checkers. In addition, it compared favourably to physicians, whereby it underperformed them under some accuracy metrics (e.g., precision and F1-measure), but outperformed them under some others (e.g., M1, M3, M5, recall, and NDCG). We will continue evolving Avey’s AI model. Furthermore, we will study its usability with real patients, examine how they respond to its suggestions, and measure its impact on their subsequent choices for care, among others.

show abstract

SERIES: eHealth in primary care. Part 5: A critical appraisal of five widely used eHealth applications for primary care – opportunities and challenges

Cited by 11 publications

References 40 publications

A Digital Lifestyle Coach (E-Supporter 1.0) to Support People With Type 2 Diabetes: Participatory Development Study

A Digital Lifestyle Coach (E-Supporter 1.0) to Support People With Type 2 Diabetes: Participatory Development Study

The Usability of Homelab, a Digital Self-service at a Dutch General Practice, for Diagnostic Tests: Pilot Study With a Questionnaire

Avey: An Accurate AI Algorithm for Self-Diagnosis

Contact Info

Product

Resources

About