A Comparison of Equal Percentile and Partial Credit Equatings for Performance‐Based Assessments Composed of Free‐Response Items

Abstract

This study compares the equal percentile (EP) and partial credit (PC) equatings for raw scores derived from performance‐based assessments composed of free‐response (open‐ended) items clustered around long reading selections or multistep mathematics problems. Data are from the Maryland School Performance Assessment Program. The results suggest that Masters (1982; Wright & Masters, 1982) partial credit model may be useful for equating examinations composed of moderately easy (or not too difficult)items sharing a first principal component with at least 25% of the total variance. This conclusion appears to hold even in the presence of some level of response dependency for the items within each cluster. Although visible discrepancies were found between PC and EP equated scores in the skewed tail of the score distributions, the direction of these discrepancies is unpredictable. Therefore, it cannot be concluded from the study that the two methods give equivalent results when the distributions are markedly skewed.

Contextual Explanations of Local Dependence in Item Clusters in a Large Scale Hands‐On Science Performance Assessment

“…Thissen, Wainer, and Wang (1994) found local dependence among free-response items that rely on common content (see also Yen, 1993) or the way the knowledge and skills required by the items are taught and learned. Existence of LID in a large scale educational performance assessment in reading and mathematics has also been observed by writers including Ferrara, Huynh, and Baghi (1997); Huynh and Ferrara (1994); and Yen (1993).…”

“…This study is based on data from 27 science tasks from the 1993 edition of the Maryland School Performance Assessment Program (MSPAP). The correlational method previously used in studies by Ferrara et al (1997) and by Huynh and Ferrara (1994) was applied to the MSPAP data to identify item clusters that display observable levels of statistical LID. (Details about this statistical method are given in a subsequent section.)…”

“…These writers suggest investigating LID by checking the magnitude of the inter-item correlations for examinees at different intervals on the test score scale. The proce- dure was used by Ferrara et al (1997) and by Huynh and Ferrara (1994) to assess LID for performance assessment items in the 1991 edition of MSPAP.…”

Classical, Generalizability, and Multifaceted Rasch Detection of Interrater Variability in Large, Sparse Data Sets

“…The result was a dominant first component (54.7%, of the total variance) with a weak second component (14.0%-a single scale would account for 1 1.1%). Thus, I concluded that the data set could be considered essentially unidimensional and that it exceeded Reckase's ( 1979) and Huynh and Ferrara's (1994) minimum requirements for successful IRT calibration. Furthermore, the solution does not support any suggestion of the existence of sections or any dependency among scales that share common names across sections.…”

Development of a Scale for Assessing the Level of Computer Familiarity of Toefl Examinees

“…Principal components 2 through 5 each account for approximately 5% to 8% of the variance. In IRT-based studies of dimensionality using principal components analysis (see, for instance, Huynh & Ferrara, 1994), such a finding is usually interpreted as providing a strong indication of a dominant first factor, justifying the use of a unidimensional IRT model.…”