Evaluation of email alerts in practice: Part 2 – validation of the information assessment method

Pluye, Pierre; Grad, Roland; Johnson‐Lafleur, Janique; Bambrick, Tara; Burnand, Bernard; Mercer, Jay; Marlow, Bernard; Campbell, Craig

doi:10.1111/j.1365-2753.2009.01313.x

Cited by 24 publications

(23 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Utility, use and usefulness of the evidence retrieved will be assessed using an adapted version of the Impact Assessment Method (IAM) [41]-[43], which was specifically developed for assessing how clinicians use information, based on the Acquisition-Cognition-Application-Outcome Model [42]-[44]. This validated six-item questionnaire takes less than one minute to complete online and will be sent by e-mail for online completion following a pre-defined automatic algorithm.…”

Section: Methodsmentioning

confidence: 99%

Increasing the quantity and quality of searching for current best evidence to answer clinical questions: protocol and intervention design of the MacPLUS FS Factorial Randomized Controlled Trials

et al. 2014

View full text Add to dashboard Cite

Background & aimsFinding current best evidence for clinical decisions remains challenging. With 3,000 new studies published every day, no single evidence-based resource provides all answers or is sufficiently updated. McMaster Premium LiteratUre Service - Federated Search (MacPLUS FS) addresses this issue by looking in multiple high quality resources simultaneously and displaying results in a one-page pyramid with the most clinically useful at the top. Yet, additional logistical and educational barriers need to be addressed to enhance point-of-care evidence retrieval. This trial seeks to test three innovative interventions, among clinicians registered to MacPLUS FS, to increase the quantity and quality of searching for current best evidence to answer clinical questions.Methods & designIn a user-centered approach, we designed three interventions embedded in MacPLUS FS: (A) a web-based Clinical Question Recorder; (B) an Evidence Retrieval Coach composed of eight short educational videos; (C) an Audit, Feedback and Gamification approach to evidence retrieval, based on the allocation of `badges’ and `reputation scores.’We will conduct a randomized factorial controlled trial among all the 904 eligible medical doctors currently registered to MacPLUS FS at the hospitals affiliated with McMaster University, Canada. Postgraduate trainees (n = 429) and clinical faculty/staff (n = 475) will be randomized to each of the three following interventions in a factorial design (A x B x C). Utilization will be continuously recorded through clinicians’ accounts that track logins and usage, down to the level of individual keystrokes. The primary outcome is the rate of searches per month per user during the six months of follow-up. Secondary outcomes, measured through the validated Impact Assessment Method questionnaire, include: utility of answers found (meeting clinicians’ information needs), use (application in practice), and perceived usefulness on patient outcomes.DiscussionBuilt on effective models for the point-of-care teaching, these interventions approach evidence retrieval as a clinical skill. If effective, they may offer the opportunity to enhance it for a large audience, at low cost, providing better access to relevant evidence across many top EBM resources in parallel.Trial registrationClinicalTrials.Gov NCT02038439.Electronic supplementary materialThe online version of this article (doi:10.1186/s13012-014-0125-9) contains supplementary material, which is available to authorized users.

show abstract

Section: Methodsmentioning

confidence: 99%

Increasing the quantity and quality of searching for current best evidence to answer clinical questions: protocol and intervention design of the MacPLUS FS Factorial Randomized Controlled Trials

et al. 2014

View full text Add to dashboard Cite

show abstract

“…In other work, we interviewed physicians who rated synopses they received on email, similar to the push component of this study. In our other work, we found that of 46 physicians, 8 (17%) said they retrieved synopses as archived email, 3 (7%) said they used a database other than Essential Evidence Plus for retrieving synopses, while 1 (2%) printed synopses for rereading [36]. In the current study, some participants reported technical problems with their PDA, making it likely that searches for synopses were occasionally done at a PC workstation rather than on their PDA.…”

Section: Discussionmentioning

confidence: 61%

Do Family Physicians Retrieve Synopses of Clinical Research Previously Read as Email Alerts?

Grad¹,

Pluye²,

Johnson‐Lafleur³

et al. 2011

J Med Internet Res

Self Cite

View full text Add to dashboard Cite

BackgroundA synopsis of new clinical research highlights important aspects of one study in a brief structured format. When delivered as email alerts, synopses enable clinicians to become aware of new developments relevant for practice. Once read, a synopsis can become a known item of clinical information. In time-pressured situations, remembering a known item may facilitate information retrieval by the clinician. However, exactly how synopses first delivered as email alerts influence retrieval at some later time is not known.ObjectivesWe examined searches for clinical information in which a synopsis previously read as an email alert was retrieved (defined as a dyad). Our study objectives were to (1) examine whether family physicians retrieved synopses they previously read as email alerts and then to (2) explore whether family physicians purposefully retrieved these synopses. MethodsWe conducted a mixed-methods study in which a qualitative multiple case study explored the retrieval of email alerts within a prospective longitudinal cohort of practicing family physicians. Reading of research-based synopses was tracked in two contexts: (1) push, meaning to read on email and (2) pull, meaning to read after retrieval from one electronic knowledge resource. Dyads, defined as synopses first read as email alerts and subsequently retrieved in a search of a knowledge resource, were prospectively identified. Participants were interviewed about all of their dyads. Outcomes were the total number of dyads and their type. ResultsOver a period of 341 days, 194 unique synopses delivered to 41 participants resulted in 4937 synopsis readings. In all, 1205 synopses were retrieved over an average of 320 days. Of the 1205 retrieved synopses, 21 (1.7%) were dyads made by 17 family physicians. Of the 1205 retrieved synopses, 6 (0.5%) were known item type dyads. However, dyads also occurred serendipitously.ConclusionIn the single knowledge resource we studied, email alerts containing research-based synopses were rarely retrieved. Our findings help us to better understand the effect of push on pull and to improve the integration of research-based information within electronic resources for clinicians.

show abstract

“…Based on the TPB, the three determinants of a clinician’s intention to perform a behavior are: his/her attitude toward performing this behavior (perceived advantages and disadvantages of performing a behavior), his/her subjective norm (normative beliefs about the social pressure to engage or not to engage in this behavior), and his/her perceived behavioral control (beliefs, based on experience, about one’s ability to adopt this behavior). The third part of the questionnaire comprises the Information Assessment Method (IAM) that serves to evaluate cognitive impacts of the information, its relevance, usefulness and expected benefits [12]. …”

Section: Methodsmentioning

confidence: 99%

“…The underlying mechanism is that tailoring intervention strategies to address barriers and strengthen facilitators related to the innovation (here Decision Box), potential adopters (here PHT) and practice environment (here primary care clinics) will result in practice change. Barriers and supports related to the innovation will be assessed with the Information Assessment Method [12], those relative to potential adopters using Ajzen's Theory of Planned Behavior (TPB) (attitude, subjective norms, perceived behavioral control) [13] and the Inter-Professional Shared Decision-Making (IP-SDM) model [14,15], and those relative to the primary care practice environment using the IP-SDM model [16]. We will use the identified barriers to tailor the Decision Box intervention before testing it in a future RCT.…”

Section: Introductionmentioning

confidence: 99%

Barriers and facilitators to implementing Decision Boxes in primary healthcare teams to facilitate shared decisionmaking: a study protocol

Giguère

Labrecque

Grad

et al. 2012

BMC Med Inform Decis Mak

Self Cite

View full text Add to dashboard Cite

BackgroundDecision Boxes are summaries of the most important benefits and harms of health interventions provided to clinicians before they meet the patient, to prepare them to help patients make informed and value-based decisions. Our objective is to explore the barriers and facilitators to using Decision Boxes in clinical practice, more precisely factors stemming from (1) the Decision Boxes themselves, (2) the primary healthcare team (PHT), and (3) the primary care practice environment.Methods/designA two-phase mixed methods study will be conducted. Eight Decision Boxes relevant to primary care, and written in both English and in French, will be hosted on a website together with a tutorial to introduce the Decision Box. The Decision Boxes will be delivered as weekly emails over a span of eight weeks to clinicians of PHTs (family physicians, residents and nurses) in five primary care clinics located across two Canadian provinces. Using a web-questionnaire, clinicians will rate each Decision Box with the Information Assessment Method (cognitive impacts, relevance, usefulness, expected benefits) and with a questionnaire based on the Theory of Planned Behavior to study the determinants of clinicians’ intention to use what they learned from that Decision Box in their patient encounter (attitude, social norm, perceived behavioral control). Web-log data will be used to monitor clinicians’ access to the website. Following the 8-week intervention, we will conduct semi-structured group interviews with clinicians and individual interviews with clinic administrators to explore contextual factors influencing the use of the Decision Boxes. Data collected from questionnaires, focus groups and individual interviews will be combined to identify factors potentially influencing implementation of Decision Boxes in clinical practice by clinicians of PHTs.ConclusionsThis project will allow tailoring of Decision Boxes and their delivery to overcome the specific barriers identified by clinicians of PHTs to improve the implementation of shared decision making in this setting.

show abstract

Evaluation of email alerts in practice: Part 2 – validation of the information assessment method

Abstract: The content relevance of seven items was supported. For three items, revisions were needed. Interviews suggested one new item. This study has yielded a 2008 version of IAM.

Cited by 24 publications

References 14 publications

Increasing the quantity and quality of searching for current best evidence to answer clinical questions: protocol and intervention design of the MacPLUS FS Factorial Randomized Controlled Trials

Increasing the quantity and quality of searching for current best evidence to answer clinical questions: protocol and intervention design of the MacPLUS FS Factorial Randomized Controlled Trials

Do Family Physicians Retrieve Synopses of Clinical Research Previously Read as Email Alerts?

Barriers and facilitators to implementing Decision Boxes in primary healthcare teams to facilitate shared decisionmaking: a study protocol

Contact Info

Product

Resources

About