Cognitive Aids in Medicine Assessment Tool (CMAT): preliminary validation of a novel tool for the assessment of emergency cognitive aids

Abstract

Summary Applying human factors principles to the design of clinical emergency guidelines is important. The UK Civil Aviation Authority uses a Checklist Assessment Tool for evaluating the content and usability of emergency drills before introduction into service on aircraft. We hypothesised that this model could be used to develop a generic medical tool. A three‐stage modified Delphi process was used to adapt the above tool for use in designing medical emergency guidelines. The resulting Cognitive aids in Medicine Assessment Tool was then used to score and rank seven published difficult airway guidelines; the scores were used to assess its validity and reliability. Pearson's rank coefficient between these scores and scores from independent assessors was 0.89 (p = 0.007). Internal consistency, as assessed by Cronbach's alpha, was 0.74, 0.96 and 0.72 for the tool's three constituent domains of physical characteristics, content and layout/format, respectively. Inter‐rater reliability, as assessed by Cohen's kappa, ranged from 0.33 to 0.72. The adoption of our tool has the potential to improve the usability of medical emergency guidelines.

‘Help! I need somebody’: getting timely assistance in clinical practice

“…Even within the required competencies set out by the GMC in 'Outcomes for Graduates', the standards suggest that learning skills for good doctor-patient communication, especially difficult communication with vulnerable patients, is prioritised over communication with colleagues during undergraduate training. The problem is that reducing communication to a list of required exchanges carries the risk of emphasising process rather than outcome [9,10]. In addition, the focusing of communication skills on community-based scenarios and the exclusion of acute care physicians from the teaching process appears to have led to the prioritisation of careful, structured exchanges in a relaxed consultation over the rapid achievement of shared understanding between professionals in an emergency.…”

Helping Babies Breathe: When Less Is More

“…Based on ILCOR Consensus on Science with Treatment Recommendations, the HBB algorithm can link directly from positive-pressure ventilation into the more advanced ILCOR algorithm when the response to bag-mask ventilation is inadequate and expertise and equipment are available [14]. In contrast, the intermediate algorithm trades the simplicity of graphic design and limited steps for multiple, text-based conditional steps and contingencies, which violate many of the principles of human factors design [15]. Decision-making will vary with the individual capacities of team members and equipment available in a particular facility, making standardized training difficult and complicating teamwork and communication.…”

Developing the NeoReadyTM: a systems approach to supporting teams during neonatal resuscitation

“…Colour is good for discrimination, but high contrast is required. 57 2. High-contrast text and background for maximum visibility.…”

“…The outcome of this evaluation revealed that the most prevalent cognitive aid is the neonatal resuscitation algorithm. 87 In Paper 1, I provide a detailed evaluation of the ILCOR algorithm and four variations by member councils against CMAT design criteria, 57 and against medical device guidelines. 33 The outcome of this study was that the ILCOR algorithm had moderate adherence to both CMAT criteria and the medical device guidelines, but that design variations reduced the adherence to these guidelines.…”

“…Limitations of the CMAT tool were discussed; specifically that the terms 'checklist' and 'cognitive aid' were used interchangeably. 57 Although checklists are one way to support and externalize cognition-and thereby reduce cognitive load-cognitive aids can take many forms other than checklists. 59 Cognitive aids can support attention and working memory, thereby reducing cognitive load and improve situation awareness.…”