The EBMT Complications and Quality of Life Working Party has developed a computer-based algorithm, the 'eGVHD App', using a user-centered design process. Accuracy was tested using a quasi-experimental crossover design with four expert-reviewed case vignettes in a convenience sample of 28 clinical professionals. Perceived usefulness was evaluated by the technology acceptance model (TAM) and User satisfaction by the Post-Study System Usability Questionnaire (PSSUQ). User experience was positive, with a median of 6 TAM points (interquartile range: 1) and beneficial median total, and subscale PSSUQ scores. The initial standard practice assessment of the vignettes yielded 65% correct results for diagnosis and 45% for scoring. The 'eGVHD App' significantly increased diagnostic and scoring accuracy to 93% (+28%) and 88% (+43%), respectively (both P o0.05). The same trend was observed in the repeated analysis of case 2: accuracy improved by using the App (+31% for diagnosis and +39% for scoring), whereas performance tended to decrease once the App was taken away. The 'eGVHD App' could dramatically improve the quality of care and research as it increased the performance of the whole user group by about 30% at the first assessment and showed a trend for improvement of individual performance on repeated case evaluation.

Graft-versus-host disease (GvHD) assessment has been shown to be a challenge for healthcare professionals, leading to the development of the eGVHD App (www.uzleuven.be/egvhd). In this study, we formally evaluated the accuracy of using the App compared to traditional assessment methods to assess GvHD. Our national multicenter randomized controlled trial involved seven Belgian transplantation centers and 78 healthcare professionals selected using a 2-stage convenience sampling approach between January and April 2017. Using a 1:1 randomization stratified by profession, healthcare professionals were assigned to use either the App (“APP”) or their usual GvHD assessment aids (“No APP”) to assess the diagnosis and severity score of 10 expert-validated clinical vignettes. Our main outcome measure was the difference in accuracy for GvHD severity scoring between both groups. The odds of being correct were 6.14 (95%CI: 2.83–13.34) and 6.29 (95%CI: 4.32–9.15) times higher in favor of the “APP” group for diagnosis and scoring, respectively (P<0.001). App-assisted GvHD severity scoring was significantly superior for both acute and chronic GvHD, with an Odds Ratio of 17.89 and 4.34 respectively (P<0.001) and showed a significantly increased inter-observer agreement compared to standard practice. Despite a mean increase of 24 minutes (95%CI: 20.45–26.97) in the time needed to score the whole GvHD test package in the “APP” group (P<0.001), usability feedback was positive. The eGVHD App shows superior GvHD assessment accuracy compared to standard practice and has the potential to improve the quality of outcome data registration in allogeneic stem cell transplantation.

No abstract

Introduction: Accurate diagnosis and severity scoring of acute and, in particular, chronic GVHD remains a challenge for clinical practice and for correct self-reporting of GVHD data to evaluate transplant outcomes. Perceived complexity and time investment issues limit the implementation of international standards and the recently updated NIH criteria for chronic GVHD. Here, we describe the development of the EBMT GVHD App, a computer/web-based algorithm-driven application to help physicians correctly diagnose and score severity of acute and chronic GVHD and improve the implementation of current standards in clinical practice and research. Methods: We applied a User Centered Design (UCD) process, through an iterative process between end-users and the development team to ensure that the App is user-friendly and efficient. A first EBMT GVHD App version (v0.0) tested an initial GVHD algorithm. An second improved prototype v1.0 was developed as a true web application (App) compatible with desktop computers and smartphones/tablets. V1.0 relies on two modules: a diagnostic module for NIH diagnostic, distinctive, common signs or proven evidence of GVHD (skin, nails, scalp/body hair, mouth, eyes, genitals, gastrointestinal tract, liver, lungs and muscles/joints), and a scoring moduleto assess severity of acute (Glucksberg and IBMTR criteria), chronic and overlap (NIH criteria) GVHD. The App v1.0 was tested by 28 hematology professionals (University Hospitals of Leuven, Belgium): 8 senior physicians, 8 junior physicians, 2 medical students and 10 data managers/research nurses; median experience in hematology 2.25 years, range 0-30, IQR 6.6, evenly distributed, by profession and seniority, to one of two groups (A and B). Usability of the App was determined for user experience and satisfaction. User experience was tested at baseline and end of study with the technology acceptance model (TAM), evaluating six Perceived Usefulness Statements rated on a 7-point Likert-like scale (1=extremely unlikely to 7=extremely likely). User satisfaction was evaluated by PSSUQ (Post-Study System Usability Questionnaire), which scores system usefulness, information quality and interface quality, on a 7-point Likert-like scale (1=strongly agree to 7=strongly disagree). App's accuracy relied on the proportion of correctly assessed clinical scenarios from 4 representative GVHD cases developed by a panel of GVHD experts as gold standard. In a quasi-experimental crossover design, professionals were invited to solve two cases either with standard paper tools or with the App, and later crossed over to use the other tool, both for the two other cases, as well as to solve again with the new tool their previous two cases. Comparisons were performed 'within groups' and 'between groups' (A vs B). Results: User experience and satisfaction were very good, with a median of 6 TAM points for user experience, and a median overall PSSUQ score of 2.2 for user satisfaction, 2.1 for System Use, 2.4 for Information Quality and 1.7 for Interface Quality. Users (70%) reported that they would be more likely to use the App on a desktop than on a mobile device. Accuracy results were only moderate with standard paper tools: 65% for diagnosis and 45% for scoring. The use of the App significantly increased diagnostic and scoring accuracy to 94% (+29%) and 88% (+43%), respectively (both p<0.001). The App also improved accuracy of individuals repeating the same clinical case (within groups) for diagnosis (+27%) and scoring (+42%), beyond a potential learning effect. From v1.0 results, an App v2.0 has been developed refining details in the algorithm, improving term description, adding a user's manual and the option of generating patient reports, which is now ready for further testing. Conclusions: The "EBMT GVHD App" is a first electronic tool to diagnose and score GVHD. Initial testing of v1.0 uniformly showed high scores for user experience and satisfaction, accurately reflected the subtle nuances of the NIH criteria, and improved significantly the accuracy of a diverse group of hematology professionals to diagnose and score severity of GVHD, compared to their practice with standard tools. Testing of v2.0 is underway to adapt layout and screen content and to address ambiguities of current guidelines. A larger study with a subsequent v3.0 is warranted in real life setting to evaluate macroscopic scalability. Disclosures Lee: Kadmon: Consultancy; Bristol-Myers Squibb: Consultancy.

No abstract

No abstract

No abstract