For two essay questions—Question 1 given to one sample ( N 1 = 100) and Question 2 to a second group ( N 2 = 100)—ratings of student performance rendered by professors of English and by professors in other disciplines were compared for reliability and concurrent validity. From the data analyses it was concluded that the reliability and validity of the ratings provided by professors outside of English departments and by professors in English departments were nearly comparable. It appeared that differences in the reliability and concurrent validity of ratings assigned to writing samples of students might be more a function of the nature of the question posed or of variations in the average ability level of examinee groups than of differences in level of expertise on the part of readers.EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 1980,40 ALTHOUGH considerable attention has been directed toward comparisons of direct or indirect assessment of writing skill (customarily involving respective use of essay or objective tests of standard written English) as evidenced by the comprehensive review of relevant literature and by the results of an empirical study following that review (Breland and Gaynor, 1979), an important concern does exist regarding the extent to which the reliability and validity of ratings of student performance on essay examinations are dependent upon the level of expertise of the reader. In particular, within the college or university setting, are judgments rendered by professors outside English departat UNIVERSITE DE MONTREAL on August 22, 2015 epm.sagepub.com Downloaded from
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.