Exploring Document Literacy: Variables Underlying the Performance of Young Adults

Kirsch, Irwin S.; Mosenthal, Peter B.

doi:10.2307/747985

Cited by 88 publications

(116 citation statements)

References 0 publications

Supporting

Mentioning

108

Contrasting

Unclassified

Order By: Relevance

“…We wish to emphasize that the missing data would not have lead to so many indistinguishable states if the cognitive characteristics of the items had been considered during the process of constructing item subsets. Scheiblechner, H. (1972 Kirsch and Mosenthal (1990). Let r and q be two states which are indistinguishable with respect to the subset of items administered.…”

Section: Discussionmentioning

confidence: 99%

“…These items were administered by trained interviewers: the examinee was handed a document, such as a page from a phone book or bus schedule, and was then asked to respond to one or two questions which required processing of at least some of the information stored in the document. The cognitive model assumed to be underlying performance in this domain was adapted from the work of Kirsch and Mosenthal (1990) who identified features of the items which were later shown to be highly correlated with the IRT difficulty parameters of the items (Sheehan and Mislevy, 1990). …”

Section: An Application To the Domain Of Document Literacymentioning

confidence: 99%

“…The data available for the application were collected in the Survey of Young Adult Literacy, a nationwide survey of literacy skills conducted by the National Assessment of Educational Progress (NAEP) in 1985. The unobservable ordinally-scaled variables assumed to be underlying performance on document processing tasks were derived from the work of Kirsch and Mosenthal (1990) who identified features of the items which were found to be highly correlated with proficiency in the domain. Finally, two new methods for analyzing the classification results are presented.…”

mentioning

confidence: 99%

See 2 more Smart Citations

A Diagnostic Classification Model for Document Processing Skills

Sheehan

Tatsuoka

Lewis

1993

ETS Research Report Series

View full text Add to dashboard Cite

This paper introduces a modification to the Rule Space diagnostic classification procedure which allows for processing of response vectors containing missing data. Rule Space is an approach to diagnostic classification which involves characterizing examinees' performances in terms of an underlying cognitive model of generalized problem‐solving skills. It has two components: (1) a procedure for determining a comprehensive set of knowledge states, where each state is characterized in terms of a unique subset of mastered skills; and (2) a procedure for classifying examinees into one or another of the specified states. The procedure for determining a comprehensive set of knowledge states is based on the Boolean descriptive function given in Tatsuoka (1991). The procedure for classifying examinees involves comparing examinees' scored response vectors to the patterns expected within each of the specified knowledge states (Tatsuoka, 1983, 1985, and 1987). Missing data is expected to be a common problem for this approach because, although the procedure for determining the comprehensive set of knowledge states requires a large pool of items, the procedure for examinee classification can be performed with smaller (less expensive) item subsets. This approach to diagnostic classification is illustrated with data collected in the Survey of Young Adult Literacy, a nationwide survey of literacy skills conducted by the National Assessment of Educational Progress (NAEP) in 1985.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: An Application To the Domain Of Document Literacymentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

A Diagnostic Classification Model for Document Processing Skills

Sheehan

Tatsuoka

Lewis

1993

ETS Research Report Series

View full text Add to dashboard Cite

show abstract

“…Items in this schema test specific or global understanding of points which have been explicitly treated in a reading passage. As noted in Kirsch and Mosenthal (1990), appropriate solution strategies depend on the degree to which the information presented in the item text shares semantic features with the information presented in the referenced reading passage. Items with large amounts of semantic overlap may be solved by matching features in the correct option to features in the referenced reading passage.…”

Section: A Three-step Estimation Proceduresmentioning

confidence: 99%

“…Although the "Simple Passage" node forms a relatively tight cluster with little unexplained variation, the "Complex Passage" node is quite disperse, with items spanning almost the full range of difficulty values. Subsequent splits of the "Complex Passage" node are defined in terms of the Degree of Correspondence variable, an item-level variable which was originally proposed by Kirsch & Mosenthal (1990).…”

Section: A Clustering Solution For the Sat Verbal Datamentioning

confidence: 99%

A TREE‐BASED APPROACH TO PROFICIENCY SCALING & DIAGNOSTIC ASSESSMENT

Sheehan

1997

ETS Research Report Series

View full text Add to dashboard Cite

A new procedure for generating instructionally-relevant diagnostic feedback is proposed. The approach involves first constructing a strong model of student proficiency and then testing whether individual students' observed item response vectors are consistent with that model. Diagnoses are specified in terms of the combinations of skills needed to score at increasingly higher levels on a tests's reported score scale. The approach is applied to the problem of developing diagnostic feedback for the SAT I Verbal Reasoning Test. Using a variation of Wright's (1977) person-fit statistic, it is shown that the estimated proficiency model accounts for 91 % of the "explainable" variation is students' observed item response vectors.Keywords: proficiency scales, diagnostic assessment A Tree-Based Approach to Proficiency Scaling & Diagnostic AssessmentThe traditional outcome of an educational test is a set of test scores reflecting the numbers of correct and incorrect responses provided by each student. While such scores may provide reliable and stable information about students' standing relative to a group, they fall short of indicating the specific patterns of skill mastery underlying students' observed item responses. Such additional information may help students and teachers better understand the meaning of test scores and the kinds of learning which might help to improve those scores.Procedures for translating observed test results into instructionally-relevant statements about students' underlying patterns of skill mastery may be designed to provide student-level . diagnostic information or group-level diagnostic information. Student-level diagnoses characterize the individual strengths and weaknesses of individual students. Group-level diagnoses characterize the strengths and weaknesses expected for students scoring at specified points on a test's reported score scale. A collection of group-level diagnoses designed to span a test's reported score range is termed a proficiency scale.Both group-and student-level diagnoses can provide useful feedback. The detailed information available from a student-level diagnosis can help human or computerized tutors design highly individualized instructional intervention. The cross-sectional view provided by a set of group-level diagnoses can be used to: (a) demonstrate that the skills tapped by a particular measurement instrument are in fact those deemed important to measure (Sheehan and Mislevy, 1990), and (b) suggest likely areas of improvement for individual students. Both types of diagnoses can also be used to inform course placement decisions.1 Procedures for generating group-level andlor student-level diagnoses have been proposed by a number of researchers. Beaton (1988) and Beaton and Allen (1992) proposed a procedure called Scale Anchoring which involved (a) identifying subsets of test items which provided superior discrimination at successive points on a test's reported score scale; and (b) asking subject-area experts to review the items and provide detailed descr...

show abstract