Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation

Kukhareva, Polina; Weir, Charlene R.; Fiol, Guilherme Del; Aarons, Gregory A.; Taft, Teresa; Schlechter, Chelsey R.; Reese, Thomas J.; Curran, Rebecca L.; Nanjo, Claude; Borbolla, Damián; Staes, Catherine J.; Morgan, Keaton; Kramer, Heidi; Stipelman, Carole H.; Shakib, Julie H.; Flynn, Michael; Kawamoto, Kensaku

doi:10.1016/j.jbi.2022.104014

Cited by 10 publications

(6 citation statements)

References 86 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This review emphasises the importance of ‘workflow alterations’ as an implementation strategy in digital health innovation. While this is not a strategy previously identified in the ERIC framework, a newly developed evaluation framework for EHR-integrated innovations emphasises the importance of having an innovation seamlessly integrate into existing clinical and information system workflows [ 12 ]. Additionally, workflow challenges have been cited as a barrier to EHR adoption [ 4 ].…”

Section: Discussionmentioning

confidence: 99%

“…In the implementation science literature, ‘implementation’ refers to the strategy of actions/methods used to facilitate behaviour change to support adoption, integration and sustainment of innovations into clinical practice [ 10 ]. Conversely, in an IT context ‘implementation’ more commonly refers to the operationalisation of an innovation [ 11 ], while in software engineering, ‘implementation’ often includes the software coding process [ 12 ]. This review aligns with the implementation science definition and is concerned with the costs associated with implementation strategies.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Implementation costs of hospital-based computerised decision support systems: a systematic review

et al. 2023

View full text Add to dashboard Cite

Background The importance of accurately costing implementation strategies is increasingly recognised within the field of implementation science. However, there is a lack of methodological guidance for costing implementation, particularly within digital health settings. This study reports on a systematic review of costing analyses conducted alongside implementation of hospital-based computerised decision support systems. Methods PubMed, Embase, Scopus and CINAHL databases were searched between January 2010 and August 2021. Two reviewers independently screened and selected original research studies that were conducted in a hospital setting, examined the implementation of a computerised decision support systems and reported implementation costs. The Expert Recommendations for Implementing Change Framework was used to identify and categorise implementation strategies into clusters. A previously published costing framework was applied to describe the methods used to measure and value implementation costs. The reporting quality of included studies was assessed using the Consolidated Health Economic Evaluation Reporting Standards checklist. Results Titles and abstracts of 1836 articles were screened, with nine articles eligible for inclusion in the review. Implementation costs were most frequently reported under the ‘evaluative and iterative strategies’ cluster, followed by ‘provide interactive assistance’. Labour was the largest implementation-related cost in the included papers, irrespective of implementation strategy. Other reported costs included consumables, durable assets and physical space, which was mostly associated with stakeholder training. The methods used to cost implementation were often unclear. There was variation across studies in the overall quality of reporting. Conclusions A relatively small number of papers have described computerised decision support systems implementation costs, and the methods used to measure and value these costs were not well reported. Priorities for future research should include establishing consistent terminology and appropriate methods for estimating and reporting on implementation costs. Trial registration The review protocol is registered with PROSPERO (ID: CRD42021272948).

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Implementation costs of hospital-based computerised decision support systems: a systematic review

et al. 2023

View full text Add to dashboard Cite

show abstract

“…This study describes a mixed methods formative evaluation which included (i1) workflow and needs assessment to inform design and workflow integration of IT tools, (2) design of the IT tools, and (3) development of implementation tracking metrics to monitor IT tools use. These evaluations were driven by a socio-technical approach known as the Evaluation in the Life Cycle of Information Technology (ELICIT) framework 31 that includes a holistic assessment of the context, the users, and the technology involved. Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework was used to guide the formative evaluation and the implementation metrics design.…”

Section: Methodsmentioning

confidence: 99%

Design and implementation of electronic health record-based tools to support a weight management program in primary care

Kukhareva,

Weir,

Cedillo

et al. 2024

JAMIA Open

Self Cite

View full text Add to dashboard Cite

Objectives This paper reports on a mixed methods formative evaluation to support the design and implementation of information technology (IT) tools for a primary care weight management intervention delivered through the patient portal using primary care staff as coaches. Methods We performed a qualitative needs assessment, designed the IT tools to support the weight management program, and developed implementation tracking metrics. Implementation tracking metrics were designed to use real world electronic health record (EHR) data. Results The needs assessment revealed IT requirements as well as barriers and facilitators to implementation of EHR-based weight management interventions in primary care. We developed implementation metrics for the IT tools. These metrics were used in weekly project team calls to make sure that project resources were allocated to areas of need. Conclusion This study identifies the important role of IT in supporting weight management through patient identification, weight and activity tracking in the patient portal, and the use of the EHR as a population management tool. An intensive multi-level implementation approach is required for successful primary care-based weight management interventions including well-designed IT tools, comprehensive involvement of clinic leadership, and implementation tracking metrics to guide the process of workflow integration. This study helps to bridge the gap between informatics and implementation by using socio-technical formative evaluation methods early in order to support the implementation of IT tools. Trial registration clinicaltrials.gov, NCT04420936. Registered June 9, 2020.

show abstract

“…ELICIT [33], TPOM [34]) to include these aspects in future evaluation studies. Add new ethical and environmental dimensions to the existing evaluation frameworks.…”

Section: How Can the One Healthmentioning

confidence: 99%

One Health: Insights from Organizational & Social, Technology Assessment and Human Factors Perspectives

Scott,

Kuziemsky,

Zhu

et al. 2023

Yearb Med Inform

View full text Add to dashboard Cite

Objectives: To offer diverse but complementary perspectives on how biomedical and health informatics can be informed by and help to achieve the vision of One Health. Methods: Overview of key considerations and critical discussion of common themes, barriers and opportunities, based on collaborative review by International Medical Informatics Association (IMIA) working group members active in related fields. Results: Health and care systems are complex sociotechnical systems that need explicit design and implementation strategies to align with the goals of One Health. The evidence-based health informatics paradigm and associated frameworks for evaluation of digital health technologies need to broaden their scope to take full account of the One Health approach. Informatics has specific contributions to make to One Health, for example by improved user experience reducing energy consumption and effective app design enhancing medication adherence. Conclusions: One Health is inherently intertwined with ergonomic, sociotechnical and evaluation perspectives in biomedical and health informatics. Health is a planetary issue that requires interdisciplinary collaborative action. The theories and principles of biomedical and health informatics offer many opportunities to transform digital health technology to better serve the One Health agenda.

show abstract

Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation

Cited by 10 publications

References 86 publications

Implementation costs of hospital-based computerised decision support systems: a systematic review

Implementation costs of hospital-based computerised decision support systems: a systematic review

Design and implementation of electronic health record-based tools to support a weight management program in primary care

One Health: Insights from Organizational & Social, Technology Assessment and Human Factors Perspectives

Contact Info

Product

Resources

About