The effect of improving task representativeness on capturing nurses’ risk assessment judgements: a comparison of written case simulations and physical simulations

Abstract

BackgroundThe validity of studies describing clinicians’ judgements based on their responses to paper cases is questionable, because - commonly used - paper case simulations only partly reflect real clinical environments. In this study we test whether paper case simulations evoke similar risk assessment judgements to the more realistic simulated patients used in high fidelity physical simulations.Methods97 nurses (34 experienced nurses and 63 student nurses) made dichotomous assessments of risk of acute deterioration on the same 25 simulated scenarios in both paper case and physical simulation settings. Scenarios were generated from real patient cases. Measures of judgement ‘ecology’ were derived from the same case records. The relationship between nurses’ judgements, actual patient outcomes (i.e. ecological criteria), and patient characteristics were described using the methodology of judgement analysis. Logistic regression models were constructed to calculate Lens Model Equation parameters. Parameters were then compared between the modeled paper-case and physical-simulation judgements.ResultsParticipants had significantly less achievement (ra) judging physical simulations than when judging paper cases. They used less modelable knowledge (G) with physical simulations than with paper cases, while retaining similar cognitive control and consistency on repeated patients. Respiration rate, the most important cue for predicting patient risk in the ecological model, was weighted most heavily by participants.ConclusionsTo the extent that accuracy in judgement analysis studies is a function of task representativeness, improving task representativeness via high fidelity physical simulations resulted in lower judgement performance in risk assessments amongst nurses when compared to paper case simulations. Lens Model statistics could prove useful when comparing different options for the design of simulations used in clinical judgement analysis. The approach outlined may be of value to those designing and evaluating clinical simulations as part of education and training strategies aimed at improving clinical judgement and reasoning.

Alternative Lens Model Equations for Dichotomous Judgments about Dichotomous Criteria

“…The mean physical simulation judgment r a increased from .503 to .535 (t(96) = 8.73, p < .001). Judgment accuracy with the manikin simulations was significantly less than with paper cases for both the dichotomous judgments (.503 vs. .552; t(96) = 2.78, p = .006) and the confidence‐adjusted judgments (.535 vs. .570; t(96) = 2.14, p = .036), as reported previously (Yang et al, ).…”

“…To use logistic regression, the applicable lens model formula includes the four component decomposition described by Cooksey (, p. 299), modified by substituting ratios of standard deviations for R e and R s correlations, as described by Stewart (, p. 115). In addition to Cooksey's () illustration, the full logistic lens model equation has been applied recently (Adderley & Thompson, ), including with the present data (Yang et al, ). The present paper is the first to compare the logistic and linear lens model equations for the same data.…”

“…Each nurse participant assessed the risk of clinical deterioration in a set of patient scenarios, as described previously (Yang & Thompson, 2011;Yang, Thompson, Hamm, Bland, & Foster, 2013), once when patients were described with cues listed on paper and again when embodied in a manikin controlled by a patient simulator computer. They stated their confidence in each assessment.…”

“…The empirical focus of judgment studies using the lens model is the size of the lens model parameters and whether the parameters and judge-ecology relations differ between conditions or groups, such as between the paper case and manikin simulation tasks (Yang, Thompson, & Bland, 2012b;Yang et al, 2013), or between nurses and students (Yang & Thompson, 2010, 2011Yang, Thompson, & Bland, 2012a). This paper's methodological question is whether those empirical comparisons differ with different approaches to producing the lens model equations.…”

A situated cognition model for clinical reasoning performance assessment: a narrative review

“…Two articles highlight how case and environment interact with the assessee to impact clinical reasoning performance and its assessment. A study [29] of nurses' ability to use vital sign cues to categorize patients as high or low risk for deterioration compared their accuracy using identical clinical scenarios presented either as written vignettes or high-fidelity simulation. The authors used judgment analysis, an approach which applies the Lens Model theory [30].…”