BackgroundCurrently, there are no binding requirements for manufacturers prescribing which information must be included in the app descriptions of health apps.ObjectiveThe aim of this study was to investigate how medical students perceive a selection of quality principles, intended for usage decisions in the app context, and establish whether the information presented in a sample of app descriptions is perceived as sufficient for facilitating an informed usage decision.MethodsA total of 123 students (mean age 24.2 years, SD 3.4) participating in a 6-week teaching module covering cardiology and pulmonology at the University of Göttingen (original enrollment 152 students, response rate 80.9%) were included. Students were asked to read 3 store description texts of cardiological or pneumological apps and initially assess whether the descriptions sufficed for a usage decision. Subsequently, they were queried on their perception of the relevance of 9 predefined quality principles, formulated for usage decisions. An appraisal of whether the app description texts contained sufficient information to satisfy these quality principles followed. By means of 20 guiding questions, participants were then asked to identify relevant information (or a lack thereof) within the descriptions. A reassessment of whether the description texts sufficed for making a usage decision ensued. A total of 343 complete datasets were obtained.ResultsA majority of the quality principles were described as “very important” and “important” for making a usage decision. When accessed via the predefined principles, students felt unable to identify sufficient information within the app descriptions in 68.81% (2124/3087) of cases. Notably, information regarding undesired effects (91.8%, 315/343), ethical soundness (90.1%, 309/343), measures taken to avert risks (89.2%, 306/343), conflicts of interest (88.3%, 303/343), and the location of data storage (87.8%, 301/343) was lacking. Following participants’ engagement with the quality principles, statistically significant changes in their assessment of whether the app descriptions sufficed for a usage decision can be seen—McNemar-Bowker test (3)=45.803919, P<.001, Cohen g=.295. In 34.1% (117/343) cases, the assessment was revised. About 3 quarters of changed assessments were seen more critically (76.9%, 90/117). Although, initially, 70% (240/343) had been considered “sufficient,” this rate was reduced to 54.2% (186/343) in the second assessment.ConclusionsIn a considerable number of app descriptions, participants were unable to locate the information necessary for making an informed usage decision. Participants’ sensitization to the quality principles led to changes in their assessment of app descriptions as a tool for usage decisions. Better transparency in app descriptions released by manufacturers and the exposure of users to quality principles could collectively form the basis for well-founded usage decisions.

Background Health apps are often used without adequately taking aspects related to their quality under consideration. This may partially be due to inadequate awareness about necessary criteria and how to prioritize them when evaluating an app. Objective The aim of this study was to introduce a method for prioritizing quality attributes in the mobile health context. To this end, physicians were asked about their assessment of nine app quality principles relevant in health contexts and their responses were used as a basis for designing a method for app prioritization. Ultimately, the goal was to aid in making better use of limited resources (eg, time) by assisting with the decision as to the specific quality principles that deserve priority in everyday medical practice and those that can be given lower priority, even in cases where the overall principles are rated similarly. Methods A total of 9503 members of two German professional societies in the field of orthopedics were invited by email to participate in an anonymous online survey over a 1-month period. Participants were asked to rate a set of nine app quality principles using a Kano survey with functional and dysfunctional (ie, positively and negatively worded) questions. The evaluation was based on the work of Kano (baseline), supplemented by a self-designed approach. Results Among the 9503 invited members, 382 completed relevant parts of the survey (return rate of 4.02%). These participants were equally and randomly assigned to two groups (test group and validation group, n=191 each). Demographic characteristics did not significantly differ between groups (all P>.05). Participants were predominantly male (328/382, 85.9%) and older than 40 years (290/382, 75.9%). Given similar ratings, common evaluation strategies for Kano surveys did not allow for conclusive prioritization of the principles, and the same was true when using the more elaborate approach of satisfaction and dissatisfaction indices following the work of Timko. Therefore, an extended, so-called “in-line-of-sight” method was developed and applied for this evaluation. Modified from the Timko method, this approach is based on a “point of view” (POV) metric, which generates a ranking coefficient. Although the principles were previously almost exclusively rated as must-be (with the exception of resource efficiency), which was not conducive to their prioritization, the new method applied from the must-be POV resulted in identical rankings for the test and validation groups: (1) legal conformity, (2) content validity, (3) risk adequacy, (4) practicality, (5) ethical soundness, (6) usability, (7) transparency, (8) technical adequacy, and (9) resource efficiency. Conclusions Established survey methodologies based on the work of Kano predominantly seek to categorize the attributes to be evaluated. The methodology presented here is an interesting option for prioritization, and enables focusing on the most important criteria, thus saving valuable time when reviewing apps for use in the medical field, even with otherwise largely similar categorization results. The extent to which this approach is applicable beyond the scenario presented herein requires further investigation.

The recently ratified Digital Healthcare Act – DVG – has paved the way, among other issues, for the prescription of health apps. The German DVG creates an entitlement to health apps for the very first time. However, this entitlement is initially limited to low-risk medical devices that have been included in the publicly accessible Register for Digital Health Applications by the German Federal Institute for Drugs and Medical Devices. Listing in the register is granted, if the manufacturer has submitted a health app and verified that it meets basic requirements for medical devices and data security as well as positive health care effects. It is questionable, whether the DVG will lead to sustainable improvements in future patient care. In order to pursue this question and align the DGOUCʼs digitization strategy closely with its basis, an online survey was conducted among the respective members on the DVGʼs content and its associated opportunities and risks. A total of 461 German-speaking, predominantly male, experienced and elderly physicians in leading positions took part in the survey. In this study, it was shown that the majority of the participating German orthopaedic and trauma surgeons is not familiar with the DVGʼs contents. Despite a fundamentally positive attitude towards digitization, scepticism about the use of “apps on prescription” and potential risks still prevails at present. The closing of ranks between medicine, IT and the legislature via interdisciplinary expert groups and the involvement of medical societies might be obligatory.

BACKGROUND Currently, there are no binding requirements for manufacturers prescribing which information must be included in the app descriptions of health apps. OBJECTIVE The aim of this study was to investigate how medical students perceive a selection of quality principles, intended for usage decisions in the app context, and establish whether the information presented in a sample of app descriptions is perceived as sufficient for facilitating an informed usage decision. METHODS A total of 123 students (mean age 24.2 years, SD 3.4) participating in a 6-week teaching module covering cardiology and pulmonology at the University of Göttingen (original enrollment 152 students, response rate 80.9%) were included. Students were asked to read 3 store description texts of cardiological or pneumological apps and initially assess whether the descriptions sufficed for a usage decision. Subsequently, they were queried on their perception of the relevance of 9 predefined quality principles, formulated for usage decisions. An appraisal of whether the app description texts contained sufficient information to satisfy these quality principles followed. By means of 20 guiding questions, participants were then asked to identify relevant information (or a lack thereof) within the descriptions. A reassessment of whether the description texts sufficed for making a usage decision ensued. A total of 343 complete datasets were obtained. RESULTS A majority of the quality principles were described as “very important” and “important” for making a usage decision. When accessed via the predefined principles, students felt unable to identify sufficient information within the app descriptions in 68.81% (2124/3087) of cases. Notably, information regarding undesired effects (91.8%, 315/343), ethical soundness (90.1%, 309/343), measures taken to avert risks (89.2%, 306/343), conflicts of interest (88.3%, 303/343), and the location of data storage (87.8%, 301/343) was lacking. Following participants’ engagement with the quality principles, statistically significant changes in their assessment of whether the app descriptions sufficed for a usage decision can be seen—McNemar-Bowker test (3)=45.803919, P<.001, Cohen g=.295. In 34.1% (117/343) cases, the assessment was revised. About 3 quarters of changed assessments were seen more critically (76.9%, 90/117). Although, initially, 70% (240/343) had been considered “sufficient,” this rate was reduced to 54.2% (186/343) in the second assessment. CONCLUSIONS In a considerable number of app descriptions, participants were unable to locate the information necessary for making an informed usage decision. Participants’ sensitization to the quality principles led to changes in their assessment of app descriptions as a tool for usage decisions. Better transparency in app descriptions released by manufacturers and the exposure of users to quality principles could collectively form the basis for well-founded usage decisions.

Background Although apps are becoming increasingly relevant in healthcare, there is limited knowledge about how healthcare professionals perceive “quality” in this context and how quality principles that can aid them in assessing health-related apps may be prioritised. The objective was to investigate physicians’ views of predefined (general) quality principles for health apps and to determine whether a ranking algorithm applied to the acquired data can provide stable results against various demographic influences and may thus be appropriate for prioritisation. Methods Participants of an online survey of members of two German professional orthopaedics associations conducted between 02/12/2019 and 02/01/2020 were asked about their perception of a set of quality principles for health apps (i.e., “practicality,” “risk adequacy,” “ethical soundness,” “legal conformity,” “content validity,” “technical adequacy,” “usability,” “resource efficiency,” and “transparency”). Structured as a Kano survey, for each principle, there were questions about its perceived relevance and opinions regarding the presence or absence of corresponding characteristics. The available data were evaluated descriptively, and a newly developed method for prioritisation of the principles was applied overall and to different demographic strata (for validation). Results Three hundred eighty-two datasets from 9503 participants were evaluated. Legal conformity, content validity, and risk adequacy filled ranks one to three, followed by practicability, ethical soundness, and usability (ranks 4 to 6). Technical adequacy, transparency, and resource efficiency ranked last (ranks 7 to 9). The ranking based on the proposed method was relatively stable, irrespective of demographic factors. The principles were seen as essential, with one exception (“resource efficiency”). Only those with little to no interest in digitisation (22/382, 5.8%) rated the nine principles indifferently. Conclusions The specified quality principles and their prioritisation can lay a foundation for future assessments of apps in the medical field. Professional societies build upon this to highlight opportunities for digital transformations in medicine and encourage their members to participate.