Distributions of Actfl Ratings by Toefl Score Ranges

Abstract

EducationalTesting Service is an Equal Opportunity/Affirmative Action Employer.Copyright© 1992 by EducationalTesting Service. All right reserved.No part of this report may be reproduced or transmitted in any form or by any means, electronicor mechanical, includingphotocopy, recording, or any information storage and retrievalsystem, without permission in writing from the publisher. Violators will be prosecutedin accordance with bothUS and international copyrightlaws.EDUCATIONAL TESTING SERVICE, ETS, the ETS logo,TOEFL,and the TOEFL logo are registered trademarksof Educational Testing Service. AbstractThe purpose of this study was to align verbal descriptions of test takers' language performance with distributions of the numerical scores they received on the three sections (Listening Comprehension, structure and written Expression, and Reading Comprehension and Vocabulary) of the Test of English as a Foreign Language (TOEFL).The study used the American council of Teaching Foreign Languages (ACTFL) descriptors as anchors for the TOEFL scores. The ACTFL Proficiency Guidelines, from which the descriptors were taken, are widely used.Qualified English as a Second Language (ESL) instructors rated their students' listening, reading, and writing proficiency using the ACTFL descriptors. Very shortly thereafter, the ratees took the TOEFL. Students' ACTFL ratings were quantified and cross-tabulated with TOEFL section scores.Results were analyzed as follows: First, the need to adjust for rater difficulty was investigated. Second, typical descriptive data were developed. Third, several schemes were used to quantify the verbal descriptors from the ACTFL Guidelines.Fourth, the ratings and test scores were correlated. Fifth, the issue of the differential validity of the measures of speaking, listening, and reading was examined. Sixth, several estimates of the reliability of the ratings were obtained. Finally, the distributions of ratings at levels of the TOEFL section scores were developed. These latter distributions can be helpful in interpreting TOEFL scores in terms of language performance. Although there was no one-to-one correspondence between the TOEFL score level and the ACTFL rating level, the correlations between ratings and scores were substantial.The Test of English as a Foreign Language (TOEFl!) was developed in 1963 by the National Council on the Testing ofEnglish as a Foreign Language, which was formed through the cooperative effort of more than thirty organizations, public and private, that were concerned with testing the English proficiency of nonnative speakers of the language applying for admission to institutions in the United States. In 1965, Educational Testing Service (ETS) and the College Board assumed joint responsibility for the program, and in 1973, a cooperative arrangement for the operation of the program was entered into by ETS, the College Board, and the Graduate Record Examinations (GRE) Board. The membership of the College Board is composed of schools, colleges, school systems, and educational asso...

Predicting the proficiency level of language learners using lexical indices

“…Comparisons are available through the Educational Testing Service (ETS) website for total scores but not for sub-scores. In establishing our three broad proficiency categories for the TOEFL scores we referred to Wendt and Woo (2009) and Boldt, Larsen-Freeman, Reed, and Courtney (1992). However, no comparisons are available between the TOEFL tests and the ACT ESL Compass test.…”

Validating a Test Designed to Assess Esl Proficiency at Lower Developmental Levels

“…In evaluating the foregoing possibility, it is pertinent to note that similar lack of discriminant validity for generally similar types of measures was reported by Boldt, Larsen-Freeman, Reed, and Courtney (1992) in an investigation of relationships between ESL teachers' ratings oflistening, writing, and reading skills based on the corresponding ACTFL schedules, on the one hand, and TOEFL section scores for listening comprehension, structure and written expression, and reading comprehension, on the other. The pattern of findings was similar to that in the present study: the level of correlation between the indirect measures (TOEFL section scores) and direct measures (teachers' ratings) under consideration was relatively strong, but the pattern of correlations for logically corresponding indirect and direct measures was not consistent with expectation.I As in the present study, the teachers involved in Boldt et al (1992), were not trained in the use of the respective rating schedules, and ratings were based on naturalistically observed samples oflinguistic behavior--that is, formally defined, standard samples of behavior from clearly delineated skill domains (e.g., writing samples, interviews, standard oral reading exercises) were not used.…”

“…The pattern of findings was similar to that in the present study: the level of correlation between the indirect measures (TOEFL section scores) and direct measures (teachers' ratings) under consideration was relatively strong, but the pattern of correlations for logically corresponding indirect and direct measures was not consistent with expectation.I As in the present study, the teachers involved in Boldt et al (1992), were not trained in the use of the respective rating schedules, and ratings were based on naturalistically observed samples oflinguistic behavior--that is, formally defined, standard samples of behavior from clearly delineated skill domains (e.g., writing samples, interviews, standard oral reading exercises) were not used.…”

Toeic®/Lpi Relationships in Academic and Employment Contexts in Thailand

“…However, scores on the arbitrarily defined scales used in reporting scores on the TOEIC (indeed, any similarly normreferenced test) do not in and of themselves permit inferences as to how well test takers who present given scale scores tend to be able to "use English," for example, for communicative purposes. As has been widely recognized (see, for example, Carroll, 1967aCarroll, , 1967bClark, 1975Clark, , 1978Clark, , 1981Woodford, 1982;Wilson, 1989;Boldt, Larsen-Freeman, Reed, & Courtney, 1992), the functional (language-use) implications of scores on norm-referenced tests can be established by correlating test scores with clearly defined "language-use" criterion variables. One such language-use criterion has featured prominently in TOEIC validation research (e.g., Woodford, 1982;Wilson, 1989;Wilson & Chavanich, 1989;Wilson & Stupak, 2001).…”

“…See Lowe and Stansfield (1988), ETS (1982), Jones (1979), Clark (1978); Carroll (1967aCarroll ( , 1967b, for historical perspective and detail regarding the "LPI model"-that is, the interview, and corresponding, behaviorally anchored scale for rating interview performance developed by the (ETS, 1982) for use in assessing the proficiency of foreign language students. See Boldt, et al (1992) for a report of research, similar conceptually to that involved in TOEIC/LPI research here under consideration, designed to link level of performance on the TOEFL to ACTFL-scale descriptors for levels of proficiency in listening, reading, and writing by correlating TOEFL scores with teachers' ratings-teachers not trained in use of the scales for rating purposes-of the respective proficiencies according to the corresponding ACTFL scales, based on naturalistic, classroom observations. Relatively strong correlations were observed in each of several university samples.…”