A Study of the Effects on Agreement Among Clinicians of Redundant and New Information, Confidence, and Time Available for Assessments

Huff, Frederick W.; Friedman, Howard

doi:10.1080/00221309.1967.9710739

“…The problem of 1The study was conducted with the assistance of a research grant provided by the Wordsworth Academy Research and Scholnrship Foundation, Fort Washington, Pa. interscorer agreement demands serious attention in light of state laws that establish critical I& scores for placement into special classes. Jastak and Jastak (1964) found the vocabulary subtests of both the WISC and WAIS t o contain items for which scoring directions were ambiguous and for which agreement among scorers was low. They found this to be the most difficult test for clinical trainees to score.…”

mentioning

confidence: 99%

“…In addition, the accuracy of the judges' diagnoses was only slightly above chance and uncorrelated with hospital-record impressions. In an investigation of factors that affect agreement among clinicians, Huff (1966Huff ( , 1967 reported that interjudge agreement was affected by almost every variable manipulated. Of the several variables tested, ratings of intelligence based on WAIS scores provided the highest reliability.Satler and Winget (1970) studied the effect of WAIS scripts on the scoring of WAIS protocols.…”

mentioning

confidence: 99%

Psychologists' scoring of WISC protocols

Miller

¹

,

Chansky

²

1972

View full text Add to dashboard Cite

The school psychologist may be regarded as a professional agent who not only measures a student's ability and interprets his scores, but also communicates a child's achievements to his parents, to his teachers and to the school administrators. Recommendations concerning a child's classroom assignment, be it to a class for educable mentally retarded, emotionally disturbed or normals, certainly will affect his destiny. Many a psychologist, with good justification, places great stock in intelligence test results. One of the consistently stable predictors of scholastic achievement has been the intelligence test score. By far, the most frequently used type of intelligence test is the paper and pencil, objectively keyed measure. This type of test is seriously limited in its ability to measure that portion of the school population that demonstrates deficits in certain motor or complex functions, such as reading. Therefore, psychologists have administered the individual test of ability to assess the marginally able student. One of the two tests popularly used for such purposes is the Wechsler Intelligence Scale for Children (WISC), the subject of the present study.Recommendations that stem from individual tests of ability are largely prognostic in nature. Like other predictiqn situations, their utility depends on the validity and reliability of the ability and achievement tests. With respect to validity, Zach (1966) brings to our attention an alternative explanation of the high ability and achievement relationship, namely, children whose obtained ability scores are low are placed in classes for slow learners and foqa variety of reasons perform poorly in class. Hence, poor performance may be due to factors confounded with low ability classroom placement and fulfills the poor performance prophecy.The focus of the present study is on the second of the two points made earlier, namely, reliability. Watson (1967a, 196713) found that psychologists showed low interjudge agreement of the Draw-A-Person Test. He reported that diagnostic impressions based on DAP alone were neither valid nor consistent between raters. In addition, the accuracy of the judges' diagnoses was only slightly above chance and uncorrelated with hospital-record impressions. In an investigation of factors that affect agreement among clinicians, Huff (1966Huff ( , 1967 reported that interjudge agreement was affected by almost every variable manipulated. Of the several variables tested, ratings of intelligence based on WAIS scores provided the highest reliability.Satler and Winget (1970) studied the effect of WAIS scripts on the scoring of WAIS protocols. Accomplices memorized responses to Comprehension, Similarities, and Vocabulary items. They found that more credit for ambiguious responses was given to Ss believed to have high IQs than to those believed to have lower IQs, which suggests a constant error in the scoring of this test. Massey (1965) urged the adoption of a more uniform scoring system for the Wechsler scales. He found that the range of scores of c...

show abstract

“…Recent psychology literature suggests that the retrieval of such interactive cue patterns requires deeper cognitive processing (Hastie, 1981;Ceci and Liker, 1986) and needs more thinking or decision time (Hogarth, 1975;Shugan, 1980;Hastie and Park, 1986). Generally, the deeper the processing, the more time is needed (Huff and Friedman, 1967;Eggleton, 1982). It follows that when auditors have less time to engage in deeper processing of their existing knowledge structures for interactive cue patterns, they will exhibit less configural information processing.…”

Section: Hypothesesmentioning

confidence: 93%

The Effect of Time Pressure on Auditors' Configural Information Processing

Choo

¹

,

Firth

²

1998

Int J Auditing

4

0

View full text Add to dashboard Cite

This research investigates the effect of time pressure on auditors' configural information processing in light of the pervasiveness of time pressure in the real world auditing situation and in light of the available cognitive psychology findings that suggest configural information processing is time dependent. We hypothesize that if auditors do not have sufficient time to retrieve specific patterns of cue interactions from their knowledge structures, they will not be in a position to invoke configural information processing. We further hypothesize that auditors will invoke a variety of cognitive simplification strategies under time pressure. The net effect of these cognitive simplification strategies is that the auditors' judgements will be made based on decreasing weights given to interactive terms. Our experiment confirms the absence of configural information processing among auditors under time pressure. The findings also directionally support the hypothesis that the variance accounted for by the interaction terms will diminish under time pressure due to cognitive simplification strategies. SUMMARYOne strand of recent auditing research has been the study of auditors' judgement processes. Experiments are conducted where auditors are asked to make judgements on audit tasks and they are given specific information cues that they can use in reaching their judgements. Models of these judgements are constructed which, among other things, attempt to identify the cues that are important in the judgement process and attempt to identify the form of the model. In many situations a priori reasoning suggests that configural information processing should be used and implicit in this expectation is that configural information processing will lead to more accurate judgements. In classic ANOVA experimental research designs, configural cue processing manifests itself in statistically significant interaction terms; that is, the interactions of the main effects.Many studies in auditing contexts have failed to find configural cue processing. Brown and Solomon (1991) argue that many of these studies were, in effect, designed such that no configural information processing should be expected and the non-significance of the interaction terms should not have been surprising. Another factor that may have inhibited configural processing in prior studies, and which may inhibit configural processing in reallife auditing, is time pressure. Configural information processing requires sophisticated logical thinking and under high time pressure, auditors resort to simplified cognitive processes. In order to test our conjecture we employed the research instrument of Brown and Solomon and manipulated three different levels of time pressure. The research instrument has been shown from a priori reasoning and empirical observation to induce configural information processing when there is no time pressure.Our experiment was conducted on 51 practising auditors and used a standard ANOVA research design. We found support for our two hypotheses, namely (1...

show abstract

“…Recent psychology literature suggests that the retrieval of such interactive cue patterns requires deeper cognitive processing (Hastie, 1981;Ceci and Liker, 1986) and needs more thinking or decision time (Hogarth, 1975;Shugan, 1980;Hastie and Park, 1986). Generally, the deeper the processing, the more time is needed (Huff and Friedman, 1967;Eggleton, 1982). It follows that when auditors have less time to engage in deeper processing of their existing knowledge structures for interactive cue patterns, they will exhibit less configural information processing.…”

Section: Hypothesesmentioning

confidence: 99%

The effect of time pressure on auditors' configural information processing

Choo

¹

,

Firth

²

1998

View full text Add to dashboard Cite

This research investigates the effect of time pressure on auditors' configural information processing in light of the pervasiveness of time pressure in the real world auditing situation and in light of the available cognitive psychology findings that suggest configural information processing is time dependent. We hypothesize that if auditors do not have sufficient time to retrieve specific patterns of cue interactions from their knowledge structures, they will not be in a position to invoke configural information processing. We further hypothesize that auditors will invoke a variety of cognitive simplification strategies under time pressure. The net effect of these cognitive simplification strategies is that the auditors' judgements will be made based on decreasing weights given to interactive terms. Our experiment confirms the absence of configural information processing among auditors under time pressure. The findings also directionally support the hypothesis that the variance accounted for by the interaction terms will diminish under time pressure due to cognitive simplification strategies. SUMMARYOne strand of recent auditing research has been the study of auditors' judgement processes. Experiments are conducted where auditors are asked to make judgements on audit tasks and they are given specific information cues that they can use in reaching their judgements. Mod-els of these judgements are constructed which, among other things, attempt to identify the cues that are important in the judgement process and attempt to identify the form of the model. In many situations a priori reasoning suggests that configural information processing should be used and implicit in this expectation is that configural information processing will lead to more accurate judgements. In classic ANOVA experimental research designs, configural cue processing manifests itself in statistically signifi-

show abstract

A Study of the Effects on Agreement Among Clinicians of Redundant and New Information, Confidence, and Time Available for Assessments

Cited by 12 publications

References 6 publications

Psychologists' scoring of WISC protocols

Psychologists' scoring of WISC protocols

The Effect of Time Pressure on Auditors' Configural Information Processing

The effect of time pressure on auditors' configural information processing

Contact Info

Product

Resources

About