Justifying the Construct Definition for a New Language Proficiency Assessment: The Redesigned<i>TOEIC Bridge®</i>Tests—Framework Paper

Schmidgall, Jonathan; Oliveri, María Elena; Duke, Trina; Grissom, Elizabeth Carter

doi:10.1002/ets2.12267

Cited by 5 publications

(19 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The speaking section included 24 statements. Seven statements were included based on their relevance to the TOEIC Bridge Speaking test construct definition, which elaborates the communication goals and linguistic skills the test measures (see Schmidgall et al, 2019). For example, one of the communication goals assessed is the ability to ask for and provide basic information; this was included as the can‐do statement, “ask for and provide basic information about everyday topics.” The remaining 17 statements were based on descriptors drawn from proficiency levels from three different language proficiency standards (ACTFL Novice High to Intermediate High, CEFR A1 to B2, CLB 1–6).…”

Section: Study 1: Speaking and Writingmentioning

confidence: 99%

“…The redesigned TOEIC Bridge® tests aim to assess the listening, reading, speaking, and writing proficiency of beginning to lower–intermediate English language learners in the context of everyday life (Schmidgall et al, 2019). For each of the four testing components (listening, reading, speaking, and writing), a construct definition was developed based on a review of theory and influential language proficiency standards.…”

mentioning

confidence: 99%

“…To further elaborate the meaning of test scores, research may also be conducted to map test scores to language proficiency standards or external measures of language proficiency (Papageorgiou et al, 2015). In the case of the redesigned TOEIC Bridge tests, influential language proficiency standards were carefully examined during the construct definition and task development phases of test design (see Everson et al, 2019; Schmidgall et al, 2019). This included the Common European Framework of Reference (CEFR) for Languages (Council of Europe, 2018), Canadian Language Benchmarks (CLB; Centre for CLB, 2012), and American Council on the Teaching of Foreign Languages (ACTFL) proficiency guidelines (ACTFL, 2012).…”

mentioning

confidence: 99%

“…This included the Common European Framework of Reference (CEFR) for Languages (Council of Europe, 2018), Canadian Language Benchmarks (CLB; Centre for CLB, 2012), and American Council on the Teaching of Foreign Languages (ACTFL) proficiency guidelines (ACTFL, 2012). These language proficiency standards include descriptors of the types of language knowledge and use that may be expected at different levels of proficiency, and TOEIC Bridge test tasks were designed to target different levels of proficiency (beginner, high beginner, low intermediate) based on a review of relevant descriptors in the CEFR, CLB, and ACTFL proficiency guidelines (Schmidgall et al, 2019). Consequently, the proposed alignment between levels of language proficiency standards and test scores may also inform expectations about additional types of activities that test takers should be able to perform at different score levels.…”

mentioning

confidence: 99%

See 3 more Smart Citations

The Redesigned TOEIC Bridge® Tests: Relations to Test‐Taker Perceptions of Proficiency in English

Schmidgall

2020

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

The redesigned four‐skills TOEIC Bridge® tests were designed to measure the listening, reading, speaking, and writing proficiency of beginning to low–intermediate English learners in the context of everyday life. In this paper, I describe two studies that were conducted to investigate claims about the meaningfulness of redesigned TOEIC Bridge test scores by comparing them to self‐assessments of listening, reading, speaking, and writing proficiency. In Study 1, test takers who participated in the redesigned TOEIC Bridge field test in Japan and Taiwan completed an online self‐assessment survey of their speaking and writing proficiency. In Study 2, test takers who participated in pretesting for the redesigned TOEIC Bridge Listening and Reading test in Japan and Taiwan completed a paper‐based self‐assessment survey of their listening and reading proficiency. Both studies were designed with the same research questions in mind in order to evaluate claims about the meaning of redesigned TOEIC Bridge test scores and to enhance the interpretation of test scores by relating them to test takers' self‐assessed ability to perform a variety of related tasks. The results indicated that self‐assessments had medium correlations with redesigned TOEIC Bridge Listening test scores (r = .55), Reading test scores (r = .54), Speaking test scores (r = .51), and Writing test scores (r = .46). These results compare favorably with the results of similar studies of the relationship between test scores and self‐assessments of language proficiency and provide evidence to support claims about the meaningfulness of TOEIC Bridge test scores. In addition, the pattern of results across TOEIC Bridge test proficiency levels and can‐do statements taken from language proficiency standards suggests that the tests meaningfully differentiate test‐takers' proficiency levels, an important consideration of test design.

show abstract

Section: Study 1: Speaking and Writingmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

The Redesigned TOEIC Bridge® Tests: Relations to Test‐Taker Perceptions of Proficiency in English

Schmidgall

2020

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

show abstract

“…How congruent are the language education experts' perspectives and the Ph.D. candidates' experiences on the MSRT-EPT? Schmidgall et al (2019) pointed out that defining the assessment construct (e.g., overall English language proficiency), which is the basis for the meaning of test scores, is one of the key steps in the test development process. However, language proficiency unique to humans in its most complex form is an abstract, invisible ability in the brain, which has nothing to do with how a test is constructed.…”

Section: Introductionmentioning

confidence: 99%

Exploring the Shortcomings of the Iranian MSRT English Proficiency Test

Ghorbani¹,

Abbassi²,

Razali³

2021

JSSH

View full text Add to dashboard Cite

The Iranian Ministry of Science, Research, and Technology (MSRT) English Proficiency Test (EPT) has been in use since 1992. While the MSRT-EPT is generally claimed to be reliable, valid, and practical, it does not assess speaking and writing skills. In this exploratory study, a qualitative approach was used to examine the MSRT-EPT test-takers experiences and language education experts’ beliefs about the test as well as their congruence with each other through semi-structured telephone interviews. Convenience and purposive sampling procedures were used to select 15 participants. Inductive coding method was applied to determine invariant constituents. Then, the constituents were reduced to categories, and finally the categories were clustered into 11 themes. Dependability and validity of the study were established through triangulation, inter-coder agreement, and member checking technique. The problems associated with the MSRT-EPT and a lack of productive skills included a lack of correspondence between the test content and Ph.D. Candidates'' needs, negative washback effect, non-theory-based content, inappropriate listening conditions, and a lack of test items originality. On the other hand, the candidates’ and experts’ perspectives were highly congruent. In light of these findings, the importance of designing a more comprehensive test including all facets of the language proficiency construct was highlighted, and some suggestions were made for future research.

show abstract

Making the Case for the Quality and Use of a New Language Proficiency Assessment: Validity Argument for the Redesigned TOEIC Bridge® Tests

Schmidgall¹,

Cid²,

Grissom³

et al. 2021

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

The redesigned TOEIC Bridge® tests were designed to evaluate test takers' English listening, reading, speaking, and writing skills in the context of everyday adult life. In this paper, we summarize the initial validity argument that supports the use of test scores for the purpose of selection, placement, and evaluation of a test taker's English skills. The validity argument consists of four major claims that provide a coherent narrative about the measurement quality and intended uses of test scores. Each major claim in the validity argument is supported by more specific claims and a summary of supporting evidence. By considering the claims and supporting evidence presented in the validity argument, readers should be able to better evaluate whether the TOEIC Bridge tests are appropriate for their situation.

show abstract

Justifying the Construct Definition for a New Language Proficiency Assessment: The RedesignedTOEIC Bridge®Tests—Framework Paper

Cited by 5 publications

References 17 publications

The Redesigned TOEIC Bridge® Tests: Relations to Test‐Taker Perceptions of Proficiency in English

The Redesigned TOEIC Bridge® Tests: Relations to Test‐Taker Perceptions of Proficiency in English

Exploring the Shortcomings of the Iranian MSRT English Proficiency Test

Making the Case for the Quality and Use of a New Language Proficiency Assessment: Validity Argument for the Redesigned TOEIC Bridge® Tests

Contact Info

Product

Resources

About