The authors examine student ratings within a new framework that emphasizes six distinct aspects of validity : content, substantive, structural, generalizability, external, and consequential. They It is fitting that as the new millennium arrives, a major revision of the Standards for Educational and Psychological Testing (American Psychological Association, 1985) has been completed and is the subject of much debate and dialogue. It reflects some changes in how the assessment community thinks about the concept of validity. The literature responsible for this shift began with Messick' s benchmark chapter on validity in the third edition of Educational Measurement (1989) and continued with Shepard' s article in the Review of Educational Research (1993) and articles on the consequences of assessment by Lane, Parke, and Stone, 1998;Linn, 1998;Reckase, 1998;Yen, 1998;Cronbach, 1989; and Moss, 1992, 1996. How validation efforts should be conducted and how assessment results are used is shifting as a consequence of this body of work.What are the implications of this work for student ratings of instruction? We take up this fundamental question in this chapter. We begin this effort by defining the concepts that are the basis for a new unified concept of validity. We briefly summarize the past validity research on student ratings using the traditional validity framework. Based on the concepts presented, we then do a critical analysis of student ratings research using the new validation framework. Synthesizing the results of the analysis, we propose an initial agenda for future student ratings' validation efforts.

Although student ratings of instruction are used to determine whether a person is teaching effectively, many people who use them are not aware of the extensive research base for them.

The purpose and methodology of evaluating teaching in higher education have changed significantly from the past.

Using both end‐of‐course achievement outcomes and long‐term cognitive retention as criteria, the present study provides comparative data on the effectiveness of a mastery and nonmastery approach to instruction. Differential effects across taxonomic levels were assessed for both criteria. The results indicated that mastery students performed significantly higher than non‐mastery students for end‐of‐course outcomes at the highest taxonomic level and equally well for knowledge, comprehension, and application level outcomes. Retention differences were found for knowledge level outcomes only, with mastery students demonstrating significantly greater retention performance. The feasibility and desirability of implementing a learning for mastery paradigm in a single course at the college level are discussed relative to the magnitude of the present results.

No abstract