Factors Influencing the Cooperative Problem-Solving of People and Computers

Abstract

It was the goal of this research to study the influence of different computer system designs on cooperative problem-solving performance. In particular, given that a computer has some knowledge about a domain, how should such knowledge be shared with the practitioner such that overall performance is improved? It was hypothesized that multiple factors contribute to performance changes, and that such factors may interact. In order to test these ideas, a formal, empirical study was conducted comparing the effectiveness of a critiquing system vs. a partially automated system when performing a medical diagnosis task. Thirty-two certified practitioners used one of the two systems to solve five test cases. The results showed that the design of the system interacted with the case characteristics and the competence level of the practitioners such that overall performance was slightly better with the partially automated system on cases where the computer's knowledge was competent (5.6% vs. 11.9% misdiagnosis rate) but on a case where the computer's knowledge was incompetent, the partially automated system induced more errors (76% vs. 43% misdiagnosis rate, p < .05). Details of the interactions causing this tradeoff in performance are discussed so that future designers may take what was learned from this study and apply it to their work.

Using the Critiquing Approach to Cope with Brittle Expert Systems

“…In his studies, however, an automation failure was potentially "obvious" to the users, since displays explicitly showed when parameters were going out of range and provided a means for the person to 'check the computer's competence. In reality, many automation failures are much more opaque than that, sometimes due to the presence of latent errors (Reason, 1991), reasoning based on incorrect, noisy, or uncertain data (Layton, Smith, and McCoy, 1994;Guerlain, 1993b) the proliferation of modes and high coupling in many such systems, (Sarter and Woods, 1994) and poor feedback as to system state (Norman, 1990). A recent study of airline pilots and dispatchers, for example, showed that in a scenario where the computer's brittleness leads to a poor recommendation, the generation of a suggestion by the computer early in the person's own problem solving can create a 30% increase in inappropriate plan selection over users of a manual version of the system (Layton et al, 1994).…”

The antibody identification assistant (AIDA), an example of a cooperative computer support system

“…The final study that will be discussed was conducted in our lab [3], [4]. Knowledge about how to rule out antibodies was encoded into a computer, called the Antibody Idenfification Assistant (ADA).…”