Quantity and quality in speaking test performance

Douglas, Dan

doi:10.1177/026553229401100203

Cited by 56 publications

(52 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Using this approach, Magnan ( 1988 ) related grammatical errors in transcriptions of 40 oral profi ciency interviews (OPI) to ratings on these interviews and found signifi cant differences in mean percentages of grammatical errors between levels of speaking profi ciency. Douglas ( 1994 ) administered the fi ve-part semidirect AGSPEAK test of speaking profi ciency to six speakers of L2 English with speakers of L1 Slovak. Test performance was rated by two trained raters on comprehensibility, grammar, and pronunciation.…”

Section: Previous Research Investigating Components Of L2 Speaking Prmentioning

confidence: 99%

Facets of Speaking Proficiency

Jong

Steinel

Florijn

et al. 2012

Stud Second Lang Acquis

173

158

View full text Add to dashboard Cite

This study examined the componential structure of second-language (L2) speaking profi ciency. Participants-181 L2 and 54 native speakers of Dutch-performed eight speaking tasks and six tasks tapping nine linguistic skills. Performance in the speaking tasks was rated on functional adequacy by a panel of judges and formed the dependent variable in subsequent analyses (structural equation modeling). The following independent variables were assessed separately: linguistic knowledge in two tests (vocabulary and grammar); linguistic processing skills (four reaction time measures obtained in three tasks: picture naming, delayed picture naming, and sentence building); and pronunciation skills (speech sounds, word stress, and intonation). All linguistic skills, with the exception of two articulation measures in the delayed picture naming task, were signifi cantly and substantially related to functional adequacy of speaking, explaining 76% of the variance. This provides substantial evidence for a componential view of L2 speaking profi ciency that consists of languageknowledge and language-processing components. The componential structure of speaking profi ciency was almost identical for the 40% of participants at the lower and the 40% of participants at the higher end of the functional adequacy distribution ( n = 73 each), which does not support Higgs and Clifford's ( 1982 ) relative contribution model, predicting that, although L2 learners become more profi cient over time, the relative weight of component skills may change.Adult native (L1) speakers differ with respect to communicative success when they speak. Some speak slowly, others fast; some articulate poorly, others well; the speech of some is characterized by a high incidence of short utterances, false starts, and self corrections, and the speech of others is characterized by long and fl awless utterances (Levelt, 1989 ). Such differences are also likely to show up in the speech of second language (L2) speakers. Compared to L1 speakers, however, L2 speakers encounter more problems in fi nding the right words, in giving their utterances the correct morphosyntactic form, and in articulating their utterances correctly and fl uently (Kormos, 2006 ;Poulisse, 1999 ). This study aims to explore the componential structure of speaking profi ciency of L2 learners at intermediate and advanced levels of L2 proficiency. In other words, this study is concerned with a fundamental issue, which has to be addressed prior to any practical matters, such as the assessment of L2 learners' profi ciency (Schoonen, 2011 ). Previous research on the components of L2 speaking profi ciency and the rationale of the current study is presented in the next two sections. Previous Research Investigating Components of L2 Speaking Profi ciencyThere have been several studies that aimed to unravel the componential structure of L2 speaking profi ciency. Two general approaches can broadly be distinguished, labeled here as the subjective-subjective and the subjective-objective approach. In the subjective...

show abstract

Section: Previous Research Investigating Components Of L2 Speaking Prmentioning

confidence: 99%

Facets of Speaking Proficiency

Jong

Steinel

Florijn

et al. 2012

Stud Second Lang Acquis

173

158

View full text Add to dashboard Cite

show abstract

“…Douglas strongly argued that raters could not arrive at the same rating for the same reasons [12]. He included six examinees on a tape-recorded test consisting of five tasks:…”

Section: Brownmentioning

confidence: 99%

“…On the other hand, other variables such as occupation, gender, and rater training did not influence test scores [7,15,19]. This implies that there might be unexamined potential variables of individual raters that influence test groups in their rating of L2 oral tests [12,16]. Additionally, rater misfit will cause a serious problem in test score interpretation and use, which is the most important issue in validity in language testing [14].…”

Section: International English Language Testing System (Ielts)mentioning

confidence: 99%

“…Another issue to consider is the research which documents that different tasks require different rating criteria [9]. Although much research has been done on this issue, there is no consensus on rating criteria for various tasks [12]. As a result, there are noticeably different contents of the rating scales across different speaking tests developed in different context.…”

Section: International English Language Testing System (Ielts)mentioning

confidence: 99%

“…We need to know more about how raters arrive at judgments. What aspects of the discourse they attend to in making their ratings, and how different arrive at similar ratings for perhaps very different reasons [12].…”

mentioning

confidence: 99%

See 2 more Smart Citations

The Diversity in an English Oral Proficiency Test

Park

2011

Journal of the Korea Academia-Industrial cooperation Society

View full text Add to dashboard Cite

There are many causes for the variation of the result in oral proficiency test such as the examiner, the task, the theme of the interview, and the gender of the participants. Previous literature documents that the rater is an important variable influencing test scores of second language oral proficiency. Although much research in language testing has been conducted concerning rater effect on test scores, there has been little attention paid to the effect of potential rater variables in language testing on their rating process. There are noticeably different contents of the rating scales across different speaking tests developed in different context. Therefore, it would not be appropriate to apply the same rating criteria for various tasks. In conclusion, we need more subject protocol analyses and more thoughtful studies on rating processes. In other words, the oral proficiency test needs a more realistic and valid tool for the assessment of second language proficiency. 요 약 영어 능력평가를 위한 테스트는 평가자, 과제, 인터뷰의 주제, 그리고 평가 받는 사람의 성별 등 여러 가지

show abstract

An Examination of Rater Orientations and Test-Taker Performance on English-for-Academic-Purposes Speaking Tasks

Brown

Iwashita

McNamara

2005

ETS Research Report Series

115

162

View full text Add to dashboard Cite

)test development efforts. As part of the foundation for the development of the next generation TOEFL test, papers and research reports were commissioned from experts within the fields of measurement, language teaching, and testing through the TOEFL 2000 project. The resulting critical reviews, expert opinions, and research results have helped to inform TOEFL program development efforts with respect to test construct, test user needs, and test delivery. Opinions expressed in these papers are those of the authors and do not necessarily reflect the views or intentions of the TOEFL program.These monographs are also of general scholarly interest, and the TOEFL program is pleased to make them available to colleagues in the fields of language teaching and testing and international student admissions in higher education.The TOEFL 2000 project was a broad effort under which language testing at Educational Testing Service ® (ETS ® ) would evolve into the 21st century. As a first step, the TOEFL program revised the Test of Spoken English™ (TSE ® ) and introduced a computer-based version of the TOEFL test. The revised TSE test, introduced in July 1995, is based on an underlying construct of communicative language ability and represents a process approach to test validation. The computer-based TOEFL test, introduced in 1998, took advantage of new forms of assessment and improved services made possible by computer-based testing, while also moving the program toward its longer-range goals, which included:• the development of a conceptual framework that takes into account models of communicative competence a research program that informs and supports this emerging framework a better understanding of the kinds of information test users need and want from the TOEFL test a better understanding of the technological capabilities for delivery of TOEFL tests into the next century Monographs 16 through 20 were the working papers that laid out the TOEFL 2000 conceptual frameworks with their accompanying research agendas. The initial framework document, Monograph 16, described the process by which the project was to move from identifying the test domain to building an empirically based interpretation of test scores. The subsequent framework documents, Monographs 17-20, extended the conceptual frameworks to the domains of reading, writing, listening, and speaking (both as independent and interdependent domains). These conceptual frameworks guided the research and prototyping studies described in subsequent monographs that resulted in the final test model. The culmination of the TOEFL 2000 project is the next generation TOEFL test that will be released in September 2005.As TOEFL 2000 projects are completed, monographs and research reports will continue to be released and public review of project work invited. TOEFL Program Educational Testing Service iii AbstractThis report documents two coordinated exploratory studies into the nature of oral English-foracademic-purposes (EAP) proficiency. Study I used verbal-report methodology to examine...

show abstract

Quantity and quality in speaking test performance

Cited by 56 publications

References 1 publication

Facets of Speaking Proficiency

Facets of Speaking Proficiency

The Diversity in an English Oral Proficiency Test

An Examination of Rater Orientations and Test-Taker Performance on English-for-Academic-Purposes Speaking Tasks

Contact Info

Product

Resources

About