Reducing problem-solving variance to improve predictability

Paul, C.J.; Acharya, Amit; Black, Bryan; Strosnider, J.K.

doi:10.1145/108515.108531

Cited by 48 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Intelligent systems are increasingly being embedded within larger host systems with demanding response requirements (Highland, 1994). Validations of embedded intelligent systems often address whether the system provides reasoning and decision support consistent with the host's decision support; support or notification when the host system slows or degrades; and decision support within a real-time response envelope (Paul, Acharya, Black, & Strosnider, 1991;Chen, Bastani, & Tsao, 1995). For some systems, system predictability is more important than sheer speed, particularly for hard real-time scheduling in process-oriented embedded systems (Cullyer, 1991).…”

Section: Eirts Evaluation Criteriamentioning

confidence: 99%

Evaluation of Embedded Intelligent Real‐time Systems*

Grabowski

Sanborn

2001

Decision Sciences

View full text Add to dashboard Cite

Over the past two decades, questions have surfaced about the effectiveness and contribution of intelligent systems to decision makers in a variety of settings. This paper focuses on the evaluation challenges associated with intelligent real-time software systems that are embedded in larger host systems. With the proliferation of such systems in operational settings such as aerospace, medical, manufacturing, and transportation systems, increased attention to evaluations of such systems, and to resulting software safety, is warranted. This paper describes one such evaluation and proposes a set of evaluation criteria for embedded intelligent real-time systems (EIRTS). Implications of the evaluation and the evaluation criteria are discussed.the masters and crew of the Exxon Benicia and other trans-Alaskan Pipeline System (TAPS) trade tankers who participated in this research. The authors would also like to thank A1 Wallace, two anonymous reviewers, and the associate editor, who contributed greatly to this manuscript. 95Evaluation of Embedded Intelligent Real-Time Systems support systems, knowledge-based systems, and expert systems. These systems were initially conceived as stand-alone information technology, designed to provide decision support to their users in a timely and appropriate fashion. Recently, such systems have evolved from stand-alone systems and integrated systems, which share resources and information with other systems, to embedded intelligent real-time systems (EIRTS), which are knowledge-based systems deployed in larger host systems with real-time response requirements. These systems are essential to the functioning of many safety-critical large-scale systems, such as air traffic control systems, worldwide financial and telecommunications systems, and real-time command and control networks.The urge to develop embedded intelligent real-time systems to guide users' actions in complex large scale systems has been described as "almost irresistible," despite questions about the effectiveness or contribution of such systems to decision makers: Given decades of research on expert systems, advisory systems, and operator aids, there are few operational success stories. Empirical studies show that advice-giving systems frequently fail to enhance overall system performance. Sometimes operators fail to request the proffered advice. Sometimes operators do not like the "tone" of the advice-giving agent and thus refuse to follow its advice. Sometimes the operators do not feel the advice is worth seeking. Other aids are effective for novices, but as operators acquire skill, the aid no longer has measurable effect (Mitchell & Sundstrom, 1997, p. 269).These observations suggest that assessments of EIRTS are important, particularly in safety-critical large-scale systems such as aviation and aerospace where software safety (Leveson, 1995) is important. In these systems, humans and technology are often jointly responsible for executing tasks and, thus, both technical and organizational evaluation measures are important....

show abstract

Section: Eirts Evaluation Criteriamentioning

confidence: 99%

Evaluation of Embedded Intelligent Real‐time Systems*

Grabowski

Sanborn

2001

Decision Sciences

View full text Add to dashboard Cite

show abstract

“…Unfortunately, many AI techniques and heuristics are not suited to analyses that would provide guaranteed response times [21]. Even when AI techniques can be shown to have predictable response times, the variance in these response times is typically so large that providing timeliness guarantees based on the worst case performance would result in severe underutilization of the computational resources during normal operations [64].…”

Section: B Real-time Artificial Intelligencementioning

confidence: 99%

Real-time computing: a new discipline of computer science and engineering

1994

View full text Add to dashboard Cite

“…Very often, however, there is uncertainty about one or both. For AI problem-solving in particular, variance in solution quality is common [11]. Because the best stopping time will vary with fluctuations in the algorithm's performance, and/or with unexpected changes of the environment, a second approach to recta-level control is to monitor the progress of the algorithm and/or the state of the environment and to determine at run-time when to stop deliberation and act on the currently available solution [6] [19].…”

Section: Introductionmentioning

confidence: 99%

Monitoring anytime algorithms

Hansen¹,

Zilberstein²

1996

SIGART Bull.

View full text Add to dashboard Cite

Anytime algorithms offer a tradeoff between solution quality and computation time that has proved useful in applying artificial intelligence techniques to time-critical problems. To exploit this tradeoff, a system must be able to determine the best time to stop deliberation and act on the currently available solution. If there is uncertainty about how much solution quality will improve with computation time, or about how the problem state may change after the start of the algorithm, monitoring the algorithm's progress and/or the problem state can make possible a better stopping decision and so improve the utility of the system. This paper analyzes the issues involved in run-time monitoring of anytime algorithms. It reviews previous work and casts the problem in a new framework from which some improved monitoring strategies emerge.1

show abstract

Reducing problem-solving variance to improve predictability

Cited by 48 publications

References 16 publications

Evaluation of Embedded Intelligent Real‐time Systems*

Evaluation of Embedded Intelligent Real‐time Systems*

Real-time computing: a new discipline of computer science and engineering

Monitoring anytime algorithms

Contact Info

Product

Resources

About