Partitioning Methods for Detecting Misconceptions of Content and Test Items

Abstract

students having different levels of sophistication with the content. One purpose of the study was to evaluate the methodology to study how items are perceived by students and to determine how students organize content. Another purpose was to determine whether sortings by members of those groups with greater sophistication would agree more with target sortings than would sortings by members of groups with less sophistication. PROCEDURE SubjectsClasses of students with varying levels of sophistication in measurement were used: high school students in grade 11, undergraduates enrolled in an educational psychology course (EPSY 200) and in a pupil evaluation course (EPSY 440), and graduate students enrolled in a pupil evaluation course (EPSY 540), an educational and psychological measurement course (EPSY 640), and in a more advanced measurement seminar (EPSY 744). For the first of two applications of the methodology (Study A), 146 students sorted the items. A second study (Study B) included 135 students who completed the sorting task. ItemsThirty multiple-choice achievement test items were used in the content areas of correlation (9 items), validity (7 items), reliability (7 items), standard error of measurement (4 items), and the relationship between reliability and validity (3 items). Item data available from previous testings were used to help protect against item faults. Also, test items were initially selected with reference to Bloom's (1956) Taxonomy of Educational Objectives: Cognitive Domain; approximately one-third of the selected items were classified as belonging to each of the first three categories, with minor 187