Using Exploratory Trials to Identify Relevant Contexts and Mechanisms in Complex Electronic Health Interventions: Evaluating the Electronic Patient-Reported Outcome Tool

Abstract

BackgroundDesigning appropriate studies for evaluating complex interventions, such as electronic health solutions to support integrated care, remains a methodological challenge. With the many moving parts of complex interventions, it is not always clear how program activities are connected to anticipated and unanticipated outcomes. Exploratory trials can be used to uncover determinants (or mechanisms) to inform content theory that underpins complex interventions before designing a full evaluation plan.ObjectiveA multimethod exploratory trial of the electronic patient-reported outcome (ePRO) tool was conducted to uncover contexts, processes and outcome variables, and the mechanisms that link these variables before full-scale evaluation. ePRO is a mobile app and portal designed to support goal-oriented care in interdisciplinary primary health care practices (clinical-level integration). This paper offers evaluation findings and methodological insight on how to use exploratory trial data to identify relevant context, process, and outcome variables, as well as central (necessary to achieving outcomes) versus peripheral (less critical and potentially context dependent) mechanisms at play.MethodsThe 4-month trial was conducted in 2 primary health care practices in Toronto, Canada. The patients were randomized into control and intervention groups and compared pre and post on quality of life and activation outcome measures. Semistructured interviews were conducted with providers and patients in the intervention group. Narrative analysis was used to uncover dominant mechanisms that inform the intervention’s content theory (how context and process variables are linked to outcomes).ResultsOverall, 7 providers, 1 administrator, and 16 patients (7-control, 9-intervention) participated in the study. This study uncovered many complex and nuanced context, process, and outcome variables at play in the intervention. Narrative analysis of patient and provider interviews revealed dominant story lines that help to tease apart central and peripheral mechanisms driving the intervention. Provider and patient story lines centered around fitting the new intervention into everyday work and life of patients and providers and meaningfulness of the intervention. These themes were moderated by patient-provider relationships going into and throughout the intervention, their comfort with technology, and the research process.ConclusionsIdentifying dominant story lines using narrative analysis helps to identify the most relevant context and process variables likely to influence study outcomes. Normalization process theory emerges as a useful theory to uncover underlying mechanisms because of its emphasis on the social production and normalization of technological, processual, and social aspects of work; all found to be critical to our intervention. The number of complex, overlapping influencing variables suggests that complex interventions such as ePRO require us to pay careful attention to central versus peripheral mechanisms that will influence s...

A mHealth-Intervention to Support the Self-Management of Multimorbid Patients in General Practices in Germany (Telepracman) – Study Protocol of a Randomized Pilot Study

“…The only comparable study showed no effects on health-related quality of life [17], and problems with integration in GP-practices work ow [18]. Nevertheless, in supplementary interviews the patients still reported increased self-con dence and motivation as a result of the intervention [17].…”

“…In order to be able to calculate a meaningful sample size for the study, recommendations refer to include at least 50 participants per arm to achieve a power of 80% and an effect size of d < 0.1 (2-side signi cance level of 5%) [34]. For the randomized trial we assumed a small effect size because studies measuring health-related quality of life with SF-12 with comparable populations showed no signi cant effects after four month [17] or showed signi cant effects at the earliest after 24 month [26]. Since the randomized study in this case has a duration of only six months, a lower effect would su ce as an indication of the effectiveness of 'TelePraCMan'.…”

“…In the research literature, we identi ed only one study on a mHealth-intervention that explicitly addressed multimorbid patients in the primary care setting [13][14][15][16][17][18]. This study showed that multimorbid patients accept and use mHealth-interventions.…”

“…This study showed that multimorbid patients accept and use mHealth-interventions. Furthermore, the intervention was able to support them in selfmanagement [17,19]. Nevertheless, there arose challenges of successfully integrating the intervention into the patients' everyday life as well as into the work ows of the GP-practices [17,18].…”

“…Furthermore, the intervention was able to support them in selfmanagement [17,19]. Nevertheless, there arose challenges of successfully integrating the intervention into the patients' everyday life as well as into the work ows of the GP-practices [17,18]. Signi cant changes in patient outcomes referring to quality of life and activation could not be shown in a fourmonths trail [17].…”