Constructing a Universal Scale of High School Course Difficulty

Abstract

This study examined the usefulness of applying the Rasch rating scale model (Andrich, 1978) to high school grade data. ACT Assessment test scores (English, Mathematics, Reading, and Science) were used as "common items" to adjust for different grading standards in individual high school courses both within and across schools. This scaling approach yielded an ACT Assessment-adjusted high school grade point average (AA-HSGPA) that was comparable across schools, cohorts, and among students within the same school and cohort who take different courses. The AA-HSGPA was constructed for all ACT-tested students (N=36,652) in 50 selected high schools. First-year college grades at a large public university were available for approximately 1,500 of these students. AA-HSGPA was a better predictor of first-year college grade point average (CGPA) than the regular high school grade point average (HSGPA). As expected, the regression of CGPA on HSGPA for high schools grouped by difficulty with regard to grading policy (easy or hard) differed, but the regressions of CGPA on AA-HSGPA and the ACT Composite score (ACTC) did not. The best model for predicting CGPA included both the ACT Composite score and AA-HSGPA. 1 Constructing a Universal Scale of High School Course DifficultyAs a measure of academic achievement, the grade point average (GPA) is limited by the extraneous effects of schools and courses. The problem of school effects was first to be recognized. Linn (1966) noted that variation in grading practices among high schools made the high school GPA (HSGPA) a sub-optimal predictor of achievement in college. The problem of course effects has received attention more recently. Students can earn a higher grade point average simply by taking easier courses. These problems fuel grade inflation at the high school level (Ziomek and Svec, 1995), and create incentives for students to avoid courses in difficult subjects such as mathematics and science (Johnson, 1997).In an effort to improve the prediction of college GPA (CGPA) by HSGPA, a number of earlier studies attempted to adjust both the predictor and criterion grades (Bashaw, 1965;Bloom and Peters, 1961;Lindquist, 1963;Potthoff, 1964;Tucker, 1963). This approach, called "central prediction systems," was an attempt to control for school effects. Disappointment with this approach centered largely on the practical difficulties of its implementation. It was difficult to find enough students from a given high school within a given college to estimate coefficients in nested regression equations. Data for one or more variables in the equations were frequently missing for many of the students who could be found. Linn (1966) noted that the accuracy of central prediction systems often was not so much better than that of simply using a standardized test, such as the ACT Assessment, in conjunction with HSGPA, to predict college achievement as to justify the cost and expense of these systems.In an attempt to construct more reliable, predictable measures of college achievement, scali...

Consistency of report card grades and external assessments in a Canadian province

“…The consistent finding is that high school grades are the better predictor of first year achievement, college graduation, and non-cognitive outcomes like leadership; prediction is optimized when grades and tests are both included in the model (Bassiri and Schulz 2003;Burton and Ramist 2001;Cliffordson 2008;D'Agostino and Bonner 2009;Willingham et al 2002;Zwick and Green 2007). Although the credibility of some of these studies is weakened by using self-report data, others include grades directly provided by high schools.…”

Intelligence and socioeconomic success: A meta-analytic review of longitudinal research

“…Future metaanalyses could, therefore, benefit from considering other variables, such as neighborhood quality (Leventhal & Brooks-Gunn, 2000), number of siblings (Blake, 1989), or parental divorce (Amato & Keith, 1991), to get a more comprehensive picture of the effects of social background. Second, the use of grade point average and class rank as the only measures of academic performance can be criticized for ignoring between-school differences; i.e., the same average grade or rank can have different meanings in different schools depending on the quality of the schools (Bassiri & Schulz, 2003). A more comprehensive study of the importance of academic performance should, therefore, also take account of the quality of the school the individual is attending.…”

Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes.

“…It has been argued that certain groups are disadvantaged by standardized test scores (Jencks & Phillips, 1998;Lawlor, Richman, & Richman, 1997;Sackett, Schmitt, Ellingson, & Kabin, 2001;Wing, 1980;Zwick, 1999). Further, high school grades do not take into account significant differences across schools in expectation and performance (Bassiri & Schulz, 2003;"In Praise of Aptitude Tests," 2005;Tam & Sukhatme, 2003) nor high school grade inflation (Ziomek & Svec, 1995). To date, standardized tests have been demonstrated as valid methods for predicting college outcomes (ACT, 1997;Willingham, Lewis, Morgan, & Ramist, 1990); however, there is general agreement that these should be augmented by the assessment of other relevant attributes (U.S. Department of Education, Office of Civil Rights, 2000).…”