Brittleness in the design of cooperative problem-solving systems: the effects on user performance

Smith, Philip J.; McCoy, Candace; Layton, Charles

doi:10.1109/3468.568744

Cited by 125 publications

(67 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been described as an unjustified assumption of satisfaction in the situation faced, although some improvements could apply (Layton et al, 1994;Parasuraman et al, 1993;Smith et al, 1997). This phenomenon is related to a low level of suspicion and is often correlated with overtrust in automation.…”

Section: Definition Of Complacencymentioning

confidence: 99%

“…Smith et al (1997) studied the use of replanning software capable of proposing new flight plans in order to avoid stormy weather. They suggested that complacency could be prevented by offering the human operators (in their case, airline pilots) several alternatives (instead of only one) from among which to choose.…”

Section: The Specific Case Of Schedulingmentioning

confidence: 99%

“…This failure has been termed complacency (Hoc, 2001;Parasuraman, Molloy, & Singh, 1993;Smith, McCoy, & Layton, 1997) and has been mainly studied in supervision tasks that imply automation or computers (for example, car driving: Hoc et al, 2006;airline pilots: Layton, Smith, & McCoy, 1994; air traffic control: Metzger & Parasuraman, 2001). However, other tasks that are only partially automated, such as those found in manufacturing scheduling, are also affected by this problem.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The role of algorithm and result comprehensibility of automated scheduling on complacency

Cegarra¹,

Hoc²

2008

Hum Ftrs & Erg Mfg Svc

View full text Add to dashboard Cite

Several studies have stressed that even expert operators who are aware of a machine's limits could adopt its proposals without questioning them (i.e., the complacency phenomenon). In production scheduling for manufacturing, this is a significant problem, as it is often suggested that the machine be allowed to build the production schedule, confining the human role to that of rescheduling. This article evaluates the characteristics of scheduling algorithms on human rescheduling performance, the quality of which was related to complacency. It is suggested that scheduling algorithms be characterized as having result comprehensibility (the result respects the scheduler's expectations in terms of the discourse rules of the information display) or algorithm comprehensibility (the complexity of the algorithm hides some important constraints). The findings stress, on the one hand, that result comprehensibility is necessary to achieve good production performance and to limit complacency. On the other hand, algorithm comprehensibility leads to poor performance due to the very high cost of understanding the algorithm. C 2008 Wiley Periodicals, Inc.

show abstract

Section: Definition Of Complacencymentioning

confidence: 99%

Section: The Specific Case Of Schedulingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The role of algorithm and result comprehensibility of automated scheduling on complacency

Cegarra¹,

Hoc²

2008

Hum Ftrs & Erg Mfg Svc

View full text Add to dashboard Cite

show abstract

“…Brittleness of machine processing, particularly in complex, high-consequence domains, is a serious issue in the design of cognitive systems (e.g., Smith, McCoy, and Layton, 1997). Approaches such as filters, summarizers, and automated search term selectors (e.g., Maes, 1998, Marx and Schmandt, 1996, Brann, Thurman, and Mitchell, 1996 are strongly committed to the machine processing being correct.…”

Section: Workload Bottleneckmentioning

confidence: 99%

Can We Ever Escape from Data Overload? A Cognitive Systems Diagnosis

Woods

Patterson

Roth³

2002

Cognition, Technology & Work

170

View full text Add to dashboard Cite

Data overload is a generic and tremendously difficult problem that has only grown with each new wave of technological capabilities. As a generic and persistent problem, three observations are in need of explanation: Why is data overload so difficult to address? Why has each wave of technology exacerbated, rather than resolved, data overload? How are people, as adaptive responsible agents in context, able to cope with the challenge of data overload?In this paper, first we examine three different characterizations that have been offered to capture the nature of the data overload problem and how they lead to different proposed solutions. As a result, we propose that (a) data overload is difficult because of the context sensitivity problem -meaning lies, not in data, but in relationships of data to interests and expectations and (b) new waves of technology exacerbate data overload when they ignore or try to finesse context sensitivity. The paper then summarizes the mechanisms of human perception and cognition that enable people to focus on the relevant subset of the available data despite the fact that what is interesting depends on context.By focusing attention on the root issues that make data overload a difficult problem and on people's fundamental competence, we have identified a set of constraints that all potential solutions must meet. Notably among these constraints is the idea that organization precedes selectivity. These constraints point toward regions of the solution space that have been little explored. In order to place data in context, designers need to display data in a conceptual space that depicts the relationships, events, and contrasts that are informative in a field of practice.KEYWORDS: agent, alarm, context, data overload, information visualization, workload Cognition, Technology and Work, in press 3 DATA OVERLOAD IS A GENERIC, DIFFICULT PROBLEM Information is not a scarce resource. Attention is.Herbert Simon 1Each round of technical advances, whether in artificial intelligence, computer graphics, or electronic connectivity promises to help people better understand and manage a whole host of activities from financial analysis to monitoring data from space missions to controlling the national air space. Certainly, this ubiquitous computerization of the modern world has tremendously advanced our ability to collect, transmit and transform data producing unprecedented levels of access to data.However, our ability to interpret this avalanche of data, i.e., to extract meaning from artificial fields of data, has expanded much more slowly, if at all. In studies across multiple settings, we find that practitioners are bombarded with computer processed data, especially when anomalies occur. We find users lost in massive networks of computer based displays, options and modes. For example, one can find a version of the following statement in most accident investigation reports: "although all of the necessary data was physically available, it was not operationally effective. No one could assemble...

show abstract

“…As a result, human management of the automated planner is crucial, as auto-planners do not always generate accurate solutions. Though fast and able to handle complex computation better than humans, com-puter optimization algorithms are notoriously "brittle" in that they can only take into account those quantifiable variables identified in the design stages that were deemed to be critical [33].…”

Section: Introductionmentioning

confidence: 99%

Assessing operator strategies for real-time replanning of multiple unmanned vehicles

Clare

Maere

Cummings

2012

IDT

View full text Add to dashboard Cite

Abstract. Future unmanned vehicles systems will invert the operator-to-vehicle ratio so that one operator controls a decentralized network of heterogeneous unmanned vehicles. This study examines the impact of allowing an operator to adjust the rate of prompts to view automation-generated plans on system performance and operator workload. Results showed that the majority of operators chose to adjust the replan prompting rate. The initial replan prompting rate had a significant framing effect on the replan prompting rates chosen throughout a scenario. Higher initial replan prompting rates led to significantly lower system performance. Operators successfully self-regulated their task-switching behavior to moderate their workload.

show abstract

Brittleness in the design of cooperative problem-solving systems: the effects on user performance

Cited by 125 publications

References 27 publications

The role of algorithm and result comprehensibility of automated scheduling on complacency

The role of algorithm and result comprehensibility of automated scheduling on complacency

Can We Ever Escape from Data Overload? A Cognitive Systems Diagnosis

Assessing operator strategies for real-time replanning of multiple unmanned vehicles

Contact Info

Product

Resources

About