“…In general, a little bit of training helps, but gaining a lot of experience after that initial stage hardly ever does. Similar conclusions have been reached in the field of medicare (Gustafson, 1963;Kundel and LaFollette, 1972;Shortliffe et al, 1979), as well as in many other domains of expertise (Camerer and Johnson, 1991). Characteristic for the field of purchasing is that we reach even sharper conclusions: in making professional judgments many experienced professional experts perform extremely poorly.…”
In a previous experiment, we have shown that risk assessments of purchasing experts are certainly not better than that of subjects untrained in purchasing, and worse than the decisions made by formal models (J. Purchas. Supply Manage. 9 (2003) 191-198). Since both these results are rather counterintuitive, we conducted a series of experiments geared at replication and extension of these findings. These new experiments show that our previous results are robust, and reveal an additional finding that is both worrying and puzzling. It actually seems to be the case that for the purchasing decision tasks in our experiments, experts perform worse with growing experience. It therefore seems that, at least for the kinds of purchasing decisions under study, it does not make much sense to use expert judgments at all. However, we show that there is a way in which expert judgments can be used in combination with formal models to improve the predictive accuracy of purchasing predictions. In our case, superior predictions are made when we combine the prediction of a formal model with the prediction of the 'average expert', thereby combining the robust linear trends as encapsulated in the formal model with the more intuitive configural rules used by experts. We provide several explanations for this phenomenon. r
“…In general, a little bit of training helps, but gaining a lot of experience after that initial stage hardly ever does. Similar conclusions have been reached in the field of medicare (Gustafson, 1963;Kundel and LaFollette, 1972;Shortliffe et al, 1979), as well as in many other domains of expertise (Camerer and Johnson, 1991). Characteristic for the field of purchasing is that we reach even sharper conclusions: in making professional judgments many experienced professional experts perform extremely poorly.…”
In a previous experiment, we have shown that risk assessments of purchasing experts are certainly not better than that of subjects untrained in purchasing, and worse than the decisions made by formal models (J. Purchas. Supply Manage. 9 (2003) 191-198). Since both these results are rather counterintuitive, we conducted a series of experiments geared at replication and extension of these findings. These new experiments show that our previous results are robust, and reveal an additional finding that is both worrying and puzzling. It actually seems to be the case that for the purchasing decision tasks in our experiments, experts perform worse with growing experience. It therefore seems that, at least for the kinds of purchasing decisions under study, it does not make much sense to use expert judgments at all. However, we show that there is a way in which expert judgments can be used in combination with formal models to improve the predictive accuracy of purchasing predictions. In our case, superior predictions are made when we combine the prediction of a formal model with the prediction of the 'average expert', thereby combining the robust linear trends as encapsulated in the formal model with the more intuitive configural rules used by experts. We provide several explanations for this phenomenon. r
“…Computer programs exist with competence comparable to human experts in areas ranging from the analysis of mass spectrograph and nuclear magnetic resonance data (Lindsay, Buchanan, Feigenbaum, & Lederberg, 1980) to the diagnosis of infectious diseases (Shortliffe, 1976). Design of an expert system for a particular task domain usually involves the interaction of "knowledge engineers" who are primarily concerned with the specification and implementation of formal problem solving techniques, and "experts" (in the relevant problem area) who provide factual and heuristic information of use for the problem solving task under consideration (Feigenbaum, 1977;Shortliffe, Buchanan, & Feigenbaum, 1979). What is still lacking, however, are theoretical frameworks capable of reducing dependence on the expert's intuition or on near exhaustive testing of possible knowledge organizations.…”
Section: Resultsmentioning
confidence: 99%
“…This analysis was guided by consideration of the disease hypotheses generated from the competitor set (TAPVC, PAPVC, ECD, and ASD), and by the "line of reasoning" (a comprehensible series of steps ;Feigenbaum, 1977;Shortliffe, Buchanan, & Feigenbaum, 1979) adopted by the subjects in attempting to reach a diagnostic conclusion.…”
Section: Analysis Of Substance Of Diagnostic Reasoning Analysis Of Tmentioning
An investigation is presented in which a computer simulation model (DIAG-NC)SER) is used to develop and test predictions for behavior of subjects in a task of medical diagnosis. The first experiment employed a process-tracing methodolagy in order to compare hypothesis generation and evaluation behavior of DIAGNOSER with individuals at different levels of expertise (students, trainees, experts). A second experiment performed with only DIAGNOSER identified conditions under which errors in reasoning in the first experiment could be related to interpretation of specific data items. Predictions derived from DIAGNOSER's performance were tested in a third experiment with a new sample of subjects. Data from the three experiments indicated that (1) form of diagnostic reasoning was similar for all subjects trained in medicine and for the simulation model, (2) substance of diagnostic reasoning employed by the simulation model was comparable with that of the more expert subjects, and (3) errors in subjects' reasoning were attributable to deficiencies in disease knowledge and the interpretation of specific patient data cues predicted by the simulation model.
“…Expert systems technology has advanced rapidly in the past few years (see, e.g., Duda and Gaschnig, 1981;Duda and Shortliffe, 1983;Gevarter, 1983b;Kinnucan, 1984;Michie, 1980;Shortliffe, Buchanan, and Feigenbaum, 1979;Webster and Miner, 1982). It is one area of artificial intelligence that appears to have come into its own and to be ready for application to the development of operational systems.…”
Recent advances in the artificial intelligence technology of knowledge-based expert systems have captivated the imaginations of designers, sponsors, and suppliers of computer-based systems in government and industry as well as researchers in university and non-profit laboratories where the technology originated. An expert system is essentially a way to capture the knowledge and expertise of a subject-matter expert and transfer it to a computer program in hopes of creating an “intelligent” computer system that will emulate the problem-solving and decision-making performance of the expert. Such systems are being built to serve as intelligent advisors and decision aids in a wide variety of application areas. We discuss conceptual issues underlying expert system design, with references to current psychological and artificial intelligence literature, and urge Consideration of these issues before undertaking development of expert systems.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.