Proficiency Scaling Based on Conditional Probability Functions for Attributes

Tatsuoka, Kikumi K.; Birenbaum, Menucha; Lewis, Charles; Sheehan, Kathleen M.

doi:10.1002/j.2333-8504.1993.tb01561.x

Cited by 18 publications

(9 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RUPP, GARCIA, JAMIESON Although item response theory item difficulty estimates were not available, the authors believe that this does not diminish the outcome of this study because it is methodological in nature and other studies have successfully used percentage correct, or the adjusted equated delta, for item difficulty analyses (e.g., Freedle & Kostin, 1993;Nissan et al, 1996;Tatsuoka, Birenbaum, Lewis, & Sheehan, 1993).…”

mentioning

confidence: 94%

Combining Multiple Regression and CART to Understand Difficulty in Second Language Reading and Listening Comprehension Test Items

Rupp¹,

García²,

Jamieson³

2001

International Journal of Testing

View full text Add to dashboard Cite

mentioning

confidence: 94%

Combining Multiple Regression and CART to Understand Difficulty in Second Language Reading and Listening Comprehension Test Items

Rupp¹,

García²,

Jamieson³

2001

International Journal of Testing

View full text Add to dashboard Cite

“…The College Board has used the model to generate scoring reports since October 2001 for the PSAT/NMSQT (Milewski and Baron 2002). The RSM was also applied to several other standardized tests such as the SAT (Tatsuoka 1993;Guerrero 2001), TOEFL (Scott 1998), TOEIC (Buck and Tatsuoka 1998), NAEP Science Assessment (Yepes- Baraya et al 1998), GRE-Q (Tatsuoka andGallagher 1998;Tatsuoka and Boodoo 2000), TIMSS and TIMSS-R Xin et al 2004).…”

Section: Introducing the Rule Space Modelmentioning

confidence: 99%

An international comparison using a diagnostic testing model: Turkish students’ profile of mathematical skills on TIMSS-R

Dogan

Tatsuoka

2007

Educ Stud Math

View full text Add to dashboard Cite

This study illustrates how a diagnostic testing model can be used to make detailed comparisons between student populations participating in international assessments. The performance of Turkish students on the TIMSS-R mathematics test was reanalyzed with a diagnostic testing model called the Rule Space Model. First, mathematical and cognitive skills ('attributes') measured by the test were determined. One hundred sixty-two items were coded in terms of their attribute involvement, creating an incidence matrix-the Q-matrix. Using the Q-matrix and the student response data, each student's attribute mastery profile was determined. Mean attribute mastery levels of Turkish students were computed and compared to those of their American peers. It was shown that Turkish students were weak in algebra and probability/statistics. They also demonstrated poor profiles in skills such as applying rules in algebra, approximation/estimation, solving open-ended problems, recognizing patterns and relationships, and quantitative reading.

show abstract

“…The first approach is illustrated in Tatsuoka (1995) and Tatsuoka, Birenbaum, Lewis and Sheehan (1993). That is, student-level diagnoses are estimated first and group-level diagnoses are subsequently obtained by summarizing available student-level diagnoses.…”

Section: A Rationale For the Tree-based Estimation Strategymentioning

confidence: 99%

“…The estimated group-level diagnoses are specified in terms of the combinations of skills needed to solve items located at increasingly higher levels on a test's reported score scale. Tatsuoka (1995) and Tatsuoka, Birenbaum, Lewis and Sheehan (1993) outlined an approach which provides both student-and group-level diagnoses. Student-level diagnoses are generated by first hypothesizing a large number of latent skill mastery states and then using a Mahalanobis distance test (Le.…”

mentioning

confidence: 99%

A TREE‐BASED APPROACH TO PROFICIENCY SCALING & DIAGNOSTIC ASSESSMENT

Sheehan

1997

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

A new procedure for generating instructionally-relevant diagnostic feedback is proposed. The approach involves first constructing a strong model of student proficiency and then testing whether individual students' observed item response vectors are consistent with that model. Diagnoses are specified in terms of the combinations of skills needed to score at increasingly higher levels on a tests's reported score scale. The approach is applied to the problem of developing diagnostic feedback for the SAT I Verbal Reasoning Test. Using a variation of Wright's (1977) person-fit statistic, it is shown that the estimated proficiency model accounts for 91 % of the "explainable" variation is students' observed item response vectors.Keywords: proficiency scales, diagnostic assessment A Tree-Based Approach to Proficiency Scaling & Diagnostic AssessmentThe traditional outcome of an educational test is a set of test scores reflecting the numbers of correct and incorrect responses provided by each student. While such scores may provide reliable and stable information about students' standing relative to a group, they fall short of indicating the specific patterns of skill mastery underlying students' observed item responses. Such additional information may help students and teachers better understand the meaning of test scores and the kinds of learning which might help to improve those scores.Procedures for translating observed test results into instructionally-relevant statements about students' underlying patterns of skill mastery may be designed to provide student-level . diagnostic information or group-level diagnostic information. Student-level diagnoses characterize the individual strengths and weaknesses of individual students. Group-level diagnoses characterize the strengths and weaknesses expected for students scoring at specified points on a test's reported score scale. A collection of group-level diagnoses designed to span a test's reported score range is termed a proficiency scale.Both group-and student-level diagnoses can provide useful feedback. The detailed information available from a student-level diagnosis can help human or computerized tutors design highly individualized instructional intervention. The cross-sectional view provided by a set of group-level diagnoses can be used to: (a) demonstrate that the skills tapped by a particular measurement instrument are in fact those deemed important to measure (Sheehan and Mislevy, 1990), and (b) suggest likely areas of improvement for individual students. Both types of diagnoses can also be used to inform course placement decisions.1 Procedures for generating group-level andlor student-level diagnoses have been proposed by a number of researchers. Beaton (1988) and Beaton and Allen (1992) proposed a procedure called Scale Anchoring which involved (a) identifying subsets of test items which provided superior discrimination at successive points on a test's reported score scale; and (b) asking subject-area experts to review the items and provide detailed descr...

show abstract

Proficiency Scaling Based on Conditional Probability Functions for Attributes

Cited by 18 publications

References 18 publications

Combining Multiple Regression and CART to Understand Difficulty in Second Language Reading and Listening Comprehension Test Items

Combining Multiple Regression and CART to Understand Difficulty in Second Language Reading and Listening Comprehension Test Items

An international comparison using a diagnostic testing model: Turkish students’ profile of mathematical skills on TIMSS-R

A TREE‐BASED APPROACH TO PROFICIENCY SCALING & DIAGNOSTIC ASSESSMENT

Contact Info

Product

Resources

About