A Context-Driven Framework for Selecting Mental Model Elicitation Methods

Abstract

Mental models describe an internal representation of knowledge of an individual or group, which can be used to interpret interactions with their environment and provide insight into decision-making strategies and prediction of performance. There are several ways to elicit mental models and analyze them; however, there is little guidance for selecting an appropriate elicitation method. Depending on different constraints of research and desired outcomes, different elicitation methods are more appropriate than others. Three criteria were identified as useful for selecting an appropriate elicitation method. These were the interaction level with participants, the number of participants being evaluated, and the resulting level of analytical detail that is required. A process for selecting the most appropriate mental model elicitation method is herein presented. Additionally, an overview of the factors that affect the selection of the mental models, and the different types of mental models are also presented.

Towards a Context-Dependent Framework for Visualizing Mental Models

“…Analytical considerations. As detailed in Harper and Dorton (2019), certain MM elicitation methods capture different levels of information. The most basic representation of a MM includes the concepts in a domain of interest, and the relationships between those concepts.…”

“…Research questions that can be answered. Harper and Dorton (2019) identified three questions answered for eliciting a MM. These were: (1) what are the concepts/relationships that make up the mental model?…”

“…These include novice/expert studies (Furlough & Gillan, 2018;Zielinska, Welk, Mayhorn, & Murphy-Hill, 2015), team performance (Lim & Klein, 2006;Stout, Cannon-Bowers, Salas, & Milanovich, 1999;Converse, Cannon-Bowers, & Salas, 1991), training (Smith-Jentsch, Ricci, Campbell, & Zeisig, 1997;Scielzo, Fiore, Cuevas, & Salas, 2002), and human-robot/machine interaction (Smith & Gillan, 1987). However, in a recent literature review, only 13 of the 93 reviewed papers (14%) on MMs actually included a visualization or graphical representation of a MM (Harper & Dorton, 2019). Somewhat ironically, the vast majority of research on MMs does not include an actual model of any kind.…”

“…A set of turnkey visual metaphors are needed to effectively convey the disparate information included in MMs. Harper and Dorton (2019) recently developed a framework for selecting the optimal MM elicitation method based on the desired outcomes and the context of practical limitations on the work being performed. That research enumerated different elicitation methods and specific data and metadata that each method collected.…”

“…There are two main components that make up a MM: Concepts and the relationships between the concepts (Harper & Dorton, 2019). Relationships between concepts specify the structural formation of knowledge in terms of what concepts are associated together, and depending on method used to obtain the MM, can also specify the strength of the association/relationship (Furlough & Gillan, 2018).…”