The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students

Abstract

The purpose of this study was to determine the extent to which performance on the TOEFL iBT speaking section is associated with other indicators of Japanese university students’ abilities to communicate orally in an academic English environment and to determine which components of oral ability for these tasks are best assessed by TOEFL iBT. To achieve this aim, TOEFL iBT speaking scores were compared to performances on a group oral discussion, picture and graph description, and prepared oral presentation tasks, and their component scores of pronunciation, fluency, grammar/vocabulary, interactional competence, descriptive skill, delivery skill, and question answering. Participants were Japanese university students (N = 222), who were English majors in a Japanese university. Pearson product–moment correlations, corrected for attenuation, between scores on the speaking section of TOEFL iBT and the three university tasks indicated strong relationships between the TOEFL iBT speaking scores and the three university tasks and high or moderate correlations between the TOEFL iBT speaking scores and the components of oral ability. For the components of oral ability, pronunciation, fluency, and vocabulary/grammar were highly associated with TOEFL iBT speaking scores while interactional competence, descriptive skill, and delivery skill were moderately associated with TOEFL iBT speaking scores. The findings suggest that TOEFL iBT speaking scores are good overall indicators of academic oral ability and that they are better measures of pronunciation, fluency and vocabulary/grammar than they are of interactional competence, descriptive skill, and presentation delivery skill.

New and not so new methods for assessing oral communication

“…These broad definitions of oral proficiency suggest that this ability includes: 1) interactional competence; 2) appropriate use of phonology; 3) appropriate and accurate use of vocabulary and grammar; and 4) appropriate fluency. (Ockey, Koyama, Setoguchi, and Sun 2015). Other research indicates that test takers prefer real-time tasks in which they actively co-construct meaning with other interlocutors.…”

Domain Description: Validating the Interpretation of the TOEFL iBT® Speaking Scores for International Teaching Assistant Screening and Certification Purposes

“…In university settings, the TOEFL iBT, including the speaking component, has been adopted for both undergraduate‐ and graduate‐level decisions, as no strong evidence was found to support differences between academic levels and disciplines in terms of tasks relevant for course completion (Jamieson et al, ). Numerous studies validated such uses of the TOEFL iBT test with respect to various aspects of assessment, including test task design, relationships with other criteria in academic contexts, and factor structure (Bridgeman, Powers, Stone, & Mollaun, ; Ockey, Koyama, Setoguchi, & Sun, ; Sawaki, Stricker, & Oranje, ). A research agenda has also been initiated for ITA‐related uses of scores.…”

Approaches and Challenges to Assessing Oral Communication on Japanese Entrance Exams

“…Like the one-on-one oral interview, the reliability of paired/group oral tasks delivered by humans depends not only on the task type, but on the effectiveness of rater training. Research has shown that group test tasks can be moderately reliable (Bonk & Ockey, 2003;Ockey, Koyama, Setoguchi, & Sun, 2015), but, in part, because the discourse is controlled by the test takers, reliability is unlikely to be high.…”