Quality Awareness and Its Influence on the Evaluation of App Meta-Information by Physicians: Validation Study

Albrecht, Urs‐Vito; Framke, Theodor; Jan, Ute von

doi:10.2196/16442

Cited by 10 publications

(24 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, it remains that the dynamic and liberal way in which the mHealth market is organized makes it difficult to assess the quality of the tools on offer, making adoption decisions more difficult, especially from the perspective of a clinician [ 8 ]. In addition, the amount of health data generated by such apps creates a need for more comprehensive privacy regulations, in particular, around the use of personal data that are clinically and scientifically meaningful [ 9 , 10 ].…”

Section: Introductionmentioning

confidence: 99%

Understanding Clinicians’ Adoption of Mobile Health Tools: A Qualitative Review of the Most Used Frameworks

Jacob¹,

Sánchez-Vázquez²,

Ivory³

2020

JMIR Mhealth Uhealth

View full text Add to dashboard Cite

Background Although there is a push toward encouraging mobile health (mHealth) adoption to harness its potential, there are many challenges that sometimes go beyond the technology to involve other elements such as social, cultural, and organizational factors. Objective This review aimed to explore which frameworks are used the most, to understand clinicians’ adoption of mHealth as well as to identify potential shortcomings in these frameworks. Highlighting these gaps and the main factors that were not specifically covered in the most frequently used frameworks will assist future researchers to include all relevant key factors. Methods This review was an in-depth subanalysis of a larger systematic review that included research papers published between 2008 and 2018 and focused on the social, organizational, and technical factors impacting clinicians’ adoption of mHealth. The initial systematic review included 171 studies, of which 50 studies used a theoretical framework. These 50 studies are the subject of this qualitative review, reflecting further on the frameworks used and how these can help future researchers design studies that investigate the topic of mHealth adoption more robustly. Results The most commonly used frameworks were different forms of extensions of the Technology Acceptance Model (TAM; 17/50, 34%), the diffusion of innovation theory (DOI; 8/50, 16%), and different forms of extensions of the unified theory of acceptance and use of technology (6/50, 12%). Some studies used a combination of the TAM and DOI frameworks (3/50, 6%), whereas others used the consolidated framework for implementation research (3/50, 6%) and sociotechnical systems (STS) theory (2/50, 4%). The factors cited by more than 20% of the studies were usefulness, output quality, ease of use, technical support, data privacy, self-efficacy, attitude, organizational inner setting, training, leadership engagement, workload, and workflow fit. Most factors could be linked to one framework or another, but there was no single framework that could adequately cover all relevant and specific factors without some expansion. Conclusions Health care technologies are generally more complex than tools that address individual user needs as they usually support patients with comorbidities who are typically treated by multidisciplinary teams who might even work in different health care organizations. This special nature of how the health care sector operates and its highly regulated nature, the usual budget deficits, and the interdependence between health care organizations necessitate some crucial expansions to existing theoretical frameworks usually used when studying adoption. We propose a shift toward theoretical frameworks that take into account implementation challenges that factor in the complexity of the sociotechnical structure of health care organizations and the interplay between the technical, social, and organizational aspects. Our consolidated framework offers recommendations on which factors to include when investigating clinicians’ adoption of mHealth, taking into account all three aspects.

show abstract

Section: Introductionmentioning

confidence: 99%

Understanding Clinicians’ Adoption of Mobile Health Tools: A Qualitative Review of the Most Used Frameworks

Jacob¹,

Sánchez-Vázquez²,

Ivory³

2020

JMIR Mhealth Uhealth

View full text Add to dashboard Cite

show abstract

“…Mit einem verstärkten Engagement und einer an das "Peer-review-"Verfahren angelehnten Qualitätskontrolle können Apps für Anwender und Verordner inhaltlich umfangreicher und sicherer werden. Zur Bewertung reicht bereits die Berücksichtigung von Qualitätsprinzipien aus[17]. Datenschutz und rechtlicher Rahmen müssen parallel politisch abgesichert werden.…”

unclassified

Viel Potenzial, wenig Substanz

Mattigk

Bellut

Albrecht

et al. 2021

Urologe

Self Cite

View full text Add to dashboard Cite

Great potential, little substance. Overview of quality and quantity of urological apps in the German app storeBackground: The range of medical apps is broad and diverse. The previous evaluations are inconsistent and limited to the respective areas of application. Objectives: The main objective of this work is to comprehensively present, organize, and evaluate the current range of urological apps with the help of a semi-automatic retrospective app store analysis (SARASA). Methods: Application of an adaptable method based on filter processes according to predefined criteria by means of SARASA to characterize urological apps from various subject areas in the Apple App Store with subsequent manual filtering and evaluation. Results: From the original list of 34,830 apps in the "Medicine" category of the Apple App Store on 27 September 2021, 3556 apps remained after apps without a Germanlanguage store description were removed. 43 subject-specific apps remained for further analysis and description. The number of reviews, rating, topicality, urological issues, technical support and richness of content were taken into account. The two most relevant apps for each topic are presented in detail. Conclusion: SARASA offers an easy-to-use method for applying filter processes to identify apps in app stores that meet predefined, formal criteria. The highest number of apps can be classified in categories of patient information and further education and training. An app officially listed by the German Federal Institute for Drugs and Medical Products (BfArM) in the sense of a digital health application (DiGA), the costs of which are reimbursed by the health insurance companies, is available for urology in only one single case. The authors see great potential for future developments in this regard.

show abstract

“…These previous studies with medical students [ 18 ] and members of the German Society for Internal Medicine [ 19 ] showed that the participants predominantly perceived all nine of the above quality principles as important. For both studies, the data were evaluated using two (randomly assigned and equally sized) test and validation groups [ 18 , 19 ].…”

Section: Introductionmentioning

confidence: 99%

“…As a foundation for this study, we used nine basic quality principles for health apps that were previously compiled [ 16 , 17 ] and evaluated [ 18 , 19 ] in a multistep process: (1) practicality, (2) risk adequacy, (3) ethical soundness, (4) legal conformity, (5) content validity, (6) technical adequacy, (7) usability, (8) resource efficiency, and (9) transparency. Participants in both of the aforementioned evaluation studies were first requested to provide initial assessments regarding the perceived relevance of these principles.…”

Section: Introductionmentioning

confidence: 99%

“…Between each of the steps, they were asked whether or not they would consider using the respective app based on the available information. During the course of these studies [ 18 , 19 ], as participants familiarized themselves with the quality criteria, they were able to make a more confident, but increasingly critical, assessments of the apps based on the available information.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Prioritization of Quality Principles for Health Apps Using the Kano Model: Survey Study

Malinka¹,

Jan²,

Albrecht³

2022

JMIR Mhealth Uhealth

Self Cite

View full text Add to dashboard Cite

Background Health apps are often used without adequately taking aspects related to their quality under consideration. This may partially be due to inadequate awareness about necessary criteria and how to prioritize them when evaluating an app. Objective The aim of this study was to introduce a method for prioritizing quality attributes in the mobile health context. To this end, physicians were asked about their assessment of nine app quality principles relevant in health contexts and their responses were used as a basis for designing a method for app prioritization. Ultimately, the goal was to aid in making better use of limited resources (eg, time) by assisting with the decision as to the specific quality principles that deserve priority in everyday medical practice and those that can be given lower priority, even in cases where the overall principles are rated similarly. Methods A total of 9503 members of two German professional societies in the field of orthopedics were invited by email to participate in an anonymous online survey over a 1-month period. Participants were asked to rate a set of nine app quality principles using a Kano survey with functional and dysfunctional (ie, positively and negatively worded) questions. The evaluation was based on the work of Kano (baseline), supplemented by a self-designed approach. Results Among the 9503 invited members, 382 completed relevant parts of the survey (return rate of 4.02%). These participants were equally and randomly assigned to two groups (test group and validation group, n=191 each). Demographic characteristics did not significantly differ between groups (all P>.05). Participants were predominantly male (328/382, 85.9%) and older than 40 years (290/382, 75.9%). Given similar ratings, common evaluation strategies for Kano surveys did not allow for conclusive prioritization of the principles, and the same was true when using the more elaborate approach of satisfaction and dissatisfaction indices following the work of Timko. Therefore, an extended, so-called “in-line-of-sight” method was developed and applied for this evaluation. Modified from the Timko method, this approach is based on a “point of view” (POV) metric, which generates a ranking coefficient. Although the principles were previously almost exclusively rated as must-be (with the exception of resource efficiency), which was not conducive to their prioritization, the new method applied from the must-be POV resulted in identical rankings for the test and validation groups: (1) legal conformity, (2) content validity, (3) risk adequacy, (4) practicality, (5) ethical soundness, (6) usability, (7) transparency, (8) technical adequacy, and (9) resource efficiency. Conclusions Established survey methodologies based on the work of Kano predominantly seek to categorize the attributes to be evaluated. The methodology presented here is an interesting option for prioritization, and enables focusing on the most important criteria, thus saving valuable time when reviewing apps for use in the medical field, even with otherwise largely similar categorization results. The extent to which this approach is applicable beyond the scenario presented herein requires further investigation.

show abstract

Quality Awareness and Its Influence on the Evaluation of App Meta-Information by Physicians: Validation Study

Cited by 10 publications

References 19 publications

Understanding Clinicians’ Adoption of Mobile Health Tools: A Qualitative Review of the Most Used Frameworks

Understanding Clinicians’ Adoption of Mobile Health Tools: A Qualitative Review of the Most Used Frameworks

Viel Potenzial, wenig Substanz

Prioritization of Quality Principles for Health Apps Using the Kano Model: Survey Study

Contact Info

Product

Resources

About