Evaluating system utility and conceptual fit using CASSM

Abstract

There is a wealth of user-centred evaluation methods (UEMs) to support the analyst in assessing interactive systems. Many of these support detailed aspects of use-for example: Is the feedback helpful? Are labels appropriate? Is the task structure optimal? Few UEMs encourage the analyst to step back and consider how well a system supports users' conceptual understandings and system utility. In this paper, we present CASSM, a method which focuses on the quality of 'fit' between users and an interactive system. We describe the methodology of conducting a CASSM analysis and illustrate the approach with three contrasting worked examples (a robotic arm, a digital library system and a drawing tool) that demonstrate different depths of analysis. We show how CASSM can help identify redesign possibilities to improve system utility. CASSM complements established evaluation methods by focusing on conceptual structures rather than procedures. Prototype tool support for completing a CASSM analysis is provided by Cassata, an open source development.

How external and internal resources influence user action: the case of infusion devices

“…The analysis focused on eliciting key user conceptions related to infusion device practice and mismatches between user and device i.e. misfits between ''the way the user thinks and the representation implemented within the system'' (p. 394; Blandford et al 2008).…”

“…Blandford et al 2008;, which considers the extent to which a system is able to support users' conceptual understanding. In order to carry out an analysis of conceptual fit, the first step involves eliciting key user conceptions before identifying mismatches between user and device, i.e.…”

“…In order to carry out an analysis of conceptual fit, the first step involves eliciting key user conceptions before identifying mismatches between user and device, i.e. misfits between ''the way the user thinks and the representation implemented within the system'' (p. 394; Blandford et al 2008). For instance, after conducting a contextual inquiry within a law firm, Attfield and Blandford (2011) identified several issues within a current-awareness alert system that related to a lack of representation of key user concepts at the interface and system level e.g.…”

Needs’ elaboration between users, designers and project leaders: Analysis of a design process of a virtual reality-based software

“…First, the needs analysis is the first step of a design process aiming to collect [22,45] or to imagine [13,54,55] the needs, and to select and translate them into specifications for the implementation in the software. Secondly, the needs analysis corresponds to actions of the needs elaboration process that may take place throughout the design, even during the use of software [38][39][40][41]. In ergonomics and in requirement engineering, authors have proposed various methods and tools to support these two approaches of needs analysis.…”

“…Further, needs are often more easily acknowledged by the users than requirements, since they relate to their every day context, rather than to an abstract system'' [34, p.283]. Although needs have often been considered as existing ''expectations'' prior to design (i.e., desires or ideas of what could or should be fulfilled with the to-be-designed object), several authors [e.g., [38][39][40][41] have insisted on the dynamic nature of needs as a social construction. They acknowledge that needs are not given a priori and evolve through the interactions between designers and users, changing circumstances and mutual learning between users and designers [41].…”