Evaluation of clinical information systems. What can be evaluated and what cannot?

Abstract

The evaluation of clinical information systems is essential as they are increasingly used in clinical routine and may even influence patient outcome on the basis of reminder functions and decision support. Therefore we try to answer three questions in this paper: what to evaluate; how to evaluate; how to interpret the results. Those key questions lead to the discussion of goals, methods and results of evaluation studies in a common context. We will compare the objectivist and the subjectivist evaluation approach and illustrate the evaluation process itself in some detail, discussing different phases of software development and potential evaluation techniques in each phase. We use four different practical examples of evaluation studies that were conducted in various settings to demonstrate how defined evaluation goals may be achieved with a limited amount of resources. This also illustrates advantages, limitations and costs of the different evaluation methods and techniques that may be used when evaluating clinical information systems.

Validity and Reliability Study for the Usability Scale of the Electronic Records System in Terms of Nursing Functions

“…C Co on nt te en nt t v va al li id di it ty y: : In order to generate items for The Usability Scale of the Electronic Record System in terms of Nursing Functions, a detailed literature review was carried out and similar studies were examined. 5,8,12,13,[17][18][19][20][21][22][23] In addition, for generating content validity, a composition on experiences (emotions, thoughts, and behaviors) regarding the relevant topic were requested from the 56 respondents, which corresponded to nurses from every unit. The attitude components reported in the written compositions were systematically examined and were used in combination with information obtained from the literature in order to generate attitude statements.…”

“…Electronic medical records make it significantly easier for nurses to complete their daily tasks. 22,23 Electronic records increase the number and accuracy of observations via routine notifications. Organized and easy-to-access patient records facilitates generating a more comprehensive nursing care plan.…”

Minimalist knowledge representation of primary care diseases in the medrapid.info knowledge base

“…The qualitative evaluation examined the problems occurring during knowledge entry: why they occurred and how they were solved. 11 For this purpose, a logbook was kept during knowledge entry using the electronic database system Microsoft Office Access 2003 TM , whereby the problems arising were described subjectively in freestyle text form. After completing the knowledge entry process, the problems were categorised by problem areas and examined.…”

A Randomized Outpatient Trial of a Decision-Support Information Technology Tool

“…Computerized decisionsupport information technology (DSIT) tools are programs that help caregivers close gaps between knowledge and performance 1,2 using various approaches, including alerts, feedback, interpretation, prognostic tools, and diagnostic aids. Although these tools are proliferating, 3 few studies [4][5][6] have rigorously evaluated their impact on ambulatory care. The DSIT tools often escape the scrutiny applied to drugs or devices, 7,8 yet they can be costly and can cause both harm 9,10 and benefit.…”