The purpose of this study is to predict the difficulty of a large sample ( n = 213) of TOEFL reading comprehension items. A related purpose was to examine whether text and text-by-item interaction variables play a significant role in predicting item difficulty. It was argued that evidence favouring the construct validity of multiple-choice reading test formats requires signincant contributions from these particular predictor variables. Details of item predictability and construct validity were explored by evaluating two hypoth eses : 1) that multiple-choice reading comprehension tests are sensitive to 12 categories of sentential and/or discourse variables found to influence compre hension processes in the experimental literature; and 2) that many of these categories of variables identified in the first hypothesis contribute significant independent variance in predicting item difficulty. For the first hypothesis, correlational analyses confirmed the importance of 11 out of the 12 categor ies, while stepwise regression analyses, accounting for up to 58% of the variance, provided some support for the second hypothesis. The pattern of predictors showed that text and text-by-item variables accounted for most of the variance, thereby providing evidence favouring the construct validity of the TOEFL reading items.

The rule-space methodology is an application of statistical pattern-recognition techniques to diagnosing the cognitive attributes (knowledge, skills, abilities, strategies, etc.) underlying test performance. The methodology provides diagnostic information on individual test-takers on each of these attributes. We used the methodology to analyze performance on a multiple-choice reading comprehension test. Based on a literature search, we hypothesized a set of 27 potentially important attributes. We carried out 4 rulespace analyses of test results for 5,000 Japanese who had taken the reading section of the TOEIC. We modified the set of attributes, deleted attributes, and identified interactions to arrive at the most parsimonious set with the maximum explanatory power. The final analysis contained 423

The purpose of this study is to explore the relationship between a set of item characteristics and the difficulty of TOEFL ® dialogue items. Identifying characteristics that are related to item difficulty has the potential to improve the efficiency of the item-writing process The study employed 365 TOEFL dialogue items, which were coded on 49 variables, including 5 significant variables reported in Nissan, DeVincenzi, and Tang (1996). Of the 5 significant variables in Nissan et al., 3 correlated significantly with item difficulty in this study. Another 11 met a critical probability criterion. These 11 included representatives from three broad categories of variables: 2 in the category of word-level factors, 1 in the category of discourse-level factors, and 8 in the category of task-processing factors. Multiple regression analyses indicate that the variables in this study account for about 40% of the variance in item difficulty. ETS administers the TOEFL program under the general direction of a policy board that was established by, and is affiliated with, the sponsoring organizations. Members of the TOEFL Board (previously the Policy Council) represent the College Board, the GRE Board, and such institutions and agencies as graduate schools of business, junior and community colleges, nonprofit educational exchange agencies, and agencies of the United States government.A continuing program of research related to the TOEFL test is carried out in consultation with the TOEFL Committee of Examiners. Its 12 members include representatives of the TOEFL Board and distinguished English as a second language specialists from the academic community. The Committee advises the TOEFL program about research needs and, through the research subcommittee, reviews and approves proposals for funding and reports for publication. Members of the Committee of Examiners serve four-year terms at the invitation of the Board; the chair of the committee serves on the Board.Because the studies are specific to the TOEFL test and the testing program, most of the actual research is conducted by ETS staff rather than by outside researchers. Many projects require the cooperation of other institutions, however, particularly those with programs in the teaching of English as a foreign or second language and applied linguistics. Representatives of such programs who are interested in participating in or conducting TOEFL-related research are invited to contact the TOEFL program office. All TOEFL research projects must undergo appropriate ETS review to ascertain that data confidentiality will be protected. Current (2003Current ( -2004 members of the TOEFL Committee of Examiners are:

The primary goal of this project was to examine the predictability of SAT reading item difficulty (equated delta) for main idea items, and collectively, the predictability of three major reading item types: main idea, inference and explicit statement items. A secondary purpose in predicting item difficulty was to contrast the responses of high verbal and low verbal ability examinees. Primary attention was paid to studying 110 main idea reading items and their associated passages. However, additional results are reported for 285 reading items taken from 34 disclosed SAT forms which represented a wider range of reading item types. The percent variance of main idea item difficulty accounted for varied from 46% to 59% depending upon the particular analysis. The predictability of all three reading item types (n = 285) varied from 21% to 29%, depending upon the particular analysis. Details of item predictability were explored by evaluating several hypotheses. Results indicated that (1) multiple‐choice reading items are sensitive to variables similar to those reported in the experimental literature on comprehension, (2) many of these variables provide significant independent predictive information in regression analyses, (3) the placement (early versus middle of text) of relevant main idea information affects item difficulty, and (4) considerable agreement between SAT and GRE reading predictability was found. Additional results contrast the performance of high and low ability groups.

The Common Core Standards call for students to be exposed to a much greater level of text complexity than has been the norm in schools for the past forty years. Textbook publishers, teachers, and assessment developers are being asked to refocus materials and methods to ensure that students are challenged to read texts at steadily increasing complexity levels as they progress through school so that all students remain on track to achieve college and career readiness by the end of 12th grade. Although automated text analysis tools have been proposed as one method for helping educators achieve this goal, research suggests that existing tools are subject to three limitations: inadequate construct coverage; overly narrow criterion variables; and inappropriate treatment of genre effects. Modeling approaches developed to address these limitations are described. Recommended approaches are incorporated into a new text analysis system called SourceRater. Validity analyses implemented on an independent sample of texts suggest that, compared to existing approaches, SourceRater's estimates of text complexity are more reflective of the complexity classifications given in the new Standards. Implications for the development of learning progressions designed to help educators organize curriculum, instruction and assessment in reading are discussed.