Individual differences in strategy use and performance during fault diagnosis

Abstract

Background Research on causal reasoning often uses group-level data analyses that downplay individual differences and simple reasoning problems that are unrepresentative of everyday reasoning. In three empirical studies, we used an individual differences approach to investigate the cognitive processes people used in fault diagnosis, which is a complex diagnostic reasoning task. After first showing how high-level fault diagnosis strategies can be composed of simpler causal inferences, we discussed how two of these strategies—elimination and inference to the best explanation (IBE)—allow normative performance, which minimizes the number of diagnostic tests, whereas backtracking strategies are less efficient. We then investigated whether the use of normative strategies was infrequent and associated with greater fluid intelligence and positive thinking dispositions and whether normative strategies used slow, analytic processing while non-normative strategies used fast, heuristic processing. Results Across three studies and 279 participants, uses of elimination and IBE were infrequent, and most participants used inefficient backtracking strategies. Fluid intelligence positively predicted elimination and IBE use but not backtracking use. Positive thinking dispositions predicted avoidance of backtracking. After classifying participants into groups that consistently used elimination, IBE, and backtracking, we found that participants who used elimination and IBE made fewer, but slower, diagnostic tests compared to backtracking users. Conclusions Participants’ fault diagnosis performance showed wide individual differences. Use of normative strategies was predicted by greater fluid intelligence and more open-minded and engaged thinking dispositions. Elimination and IBE users made the slow, efficient responses typical of analytic processing. Backtracking users made the fast, inefficient responses suggestive of heuristic processing.

The impact of inducing troubleshooting strategies via visual aids on performance in a computerized digital network task

“…In line with the general finding that apt troubleshooters make fewer tests but take longer for each of them, while poor troubleshooters conduct more tests with shorter time intervals between them (Morris & Rouse, 1985;Rouse & Rouse, 1979;Shreeves et al, 2020), Gugerty (2007) found a strong negative correlation between the…”

“…Missing information about a system, like the number and location of faulty elements, increases uncertainty. Causal reasoning, including iterations of hypothesis generation and testing, narrows the problem space and reduces uncertainty throughout the processes of fault diagnosis and problem solving (Shreeves, Gugerty, & Moore, 2020). Usually, the operators available for diagnosing or removing the fault are routine procedures that need to be known and practiced, while for new problems they are often indeterminate (Jonassen & Hung, 2006; Morrison & Duncan, 1988).…”

“…Dealing with complex systems requires the problem solver to reduce workload and uncertainty by limiting the problem space to a manageable subset of components, or by acquiring more information about the system. To this end, apt troubleshooters apply strategic knowledge by making informative tests and approaching problems in a systematic manner (Morris & Rouse, 1985; Shreeves et al, 2020). Given financial or temporal constraints, adopting efficient strategies is crucial.…”

“…Some evidence suggests that reflective vs. impulsive thinking affects the extent to which troubleshooters work efficiently and improve with practice (Morris & Rouse, 1985). Strategy use in troubleshooting has also been linked to fluid intelligence and thinking dispositions such as openmindedness and typical intellectual engagement (Shreeves et al, 2020), as well as metacognitive processing such as executive control, self-regulation, and monitoring processes (Perez, 1991). Furthermore, troubleshooting performance is susceptible to systems' visual characteristics, size, and complexity, and time constraints, all of which variably affect demands put on cognitive processes such as working memory, processing speed, and executive function (Jonassen & Hung, 2006;Morris & Rouse, 1985;Toms & Patrick, 1989).…”

“…This points to the conjecture that elimination might be more demanding in terms of working memory resources (Shreeves et al, 2020). Working memory load is assumed to be affected by complexity as well as by visual characteristics of the troubleshooting problem, and accordingly, troubleshooters are known to benefit from visual aids and graphical representations (Morris & Rouse, 1985).…”