What interpretations can we make from scores on graphic-prompt writing (GPW) tasks? An argument-based approach to test validation

Choi, YunDeok

doi:10.1016/j.asw.2021.100523

Cited by 4 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, Weir et al (2007) discovered, that overall, IELTS test-takers' computer familiarity did not result in significant effects. Likewise, adapting Lee's (2004) questionnaire, Choi (2021) found that test-takers' preferences for CBW tests did not significantly contribute to scores on a CBW test that was designed to make placement decisions while graph familiarity did to a small extent. The discrepancy might have resulted from changes in the availability and use of computers in educational settings and/or the questionnaire instruments that had yet to be validated in either study.…”

Section: Effects Of Computer Familiarity On Cbw Test Scoresmentioning

confidence: 99%

Validity of score interpretations on an online English placement writing test

Choi

2022

Lang Test Asia

Self Cite

View full text Add to dashboard Cite

A much-debated question in the L2 assessment field is if computer familiarity should be considered a potential source of construct-irrelevant variance in computer-based writing (CBW) tests. This study aims to make a partial validity argument for an online source-based writing test (OSWT) designed for English placement testing (EPT), focusing on the explanation inference. Score interpretations on the OWST are proposed and supporting evidence is sought in terms of test-takers’ self-confidence in and preferences for CBW tests (two interrelated aspects of computer familiarity) and L2 writing ability. Ninety-seven ESL students demonstrating two different levels (higher and lower levels) of L2 writing ability studying at a US university completed the OSWT and an online questionnaire asking about their attitudes towards CBW tests. A series of statistical and thematic analyses revealed that most of the test-takers held self-confidence in and preferences for CBW tests for reasons related to previous CBW experience (e.g., familiarity with CBW, useful tools/functions available on computers) regardless of L2 writing ability. The higher-level test-takers obtained significantly higher scores on the OSWT than their lower-level counterparts. Test-takers’ preferences were a significant predictor of the OSWT scores only in the higher-level group. The findings largely support the validity of proposed score interpretations on the OSWT. Implications are discussed in terms of test fairness and the construct of CBW tests.

show abstract

Section: Effects Of Computer Familiarity On Cbw Test Scoresmentioning

confidence: 99%

Validity of score interpretations on an online English placement writing test

Choi

2022

Lang Test Asia

Self Cite

View full text Add to dashboard Cite

show abstract

“…At the same time, Professor Fulcher's work has inspired many recent works of our own in the MENA region (e.g., see McCallum & Coombe, 2020Rauf & McCallum, 2020) and that of others (e.g., Davidson & Coombe, 2019;Gebril & Eid, 2017;Hidri, 2016Hidri, , 2020, as well as other local contexts and domains of study (e.g., see studies on proficiency testing in (Quyhn, 2019)); studies of rating scale development for proficiency tests for work (e.g., Knoch et al, 2020), blog posts (e.g., Rock, 2022) and studies in countries such as England (e.g., Tavakoli & Hunter ,2018), Iran (e.g., Ghanbari & Barati, 2020), the USA (Choi, 2021), among many more; studies in language assessment literacy across Asia (e.g., Lam, 2015;Xu & Brown, 2017), South America (e.g., Giraldo, 2021), Europe (Vogt & Tsagari, 2014;Vogt et al, 2020) and Turkey (e.g., Hatipoğlu, 2015) to name a few.…”

mentioning

confidence: 99%

lee.mccallum@coventry.ac.uk Language Teaching Research Quarterly 2022, Vol. 29, 1–5 Introduction to the Special Issue in Honour of Professor Glenn Fulcher

McCallum

Coombe

Mohebbi³

2022

LTRQ

View full text Add to dashboard Cite

Language testing and assessment are diverse areas which continue to experience rapid development as fields of study. The same growth is also true when we consider the range of methods of testing and assessing student abilities that we have at our disposal. Indeed, as a glance at ‘flagship’ journals such as Language Testing (2022) and Language Assessment Quarterly (2022) will confirm, the scope of academic scholarship is now enormous, as are the many ways in which testers and assessors create, administer, score, and evaluate the tests and assessments they have at their disposal. However, despite this rapid and continuous development, at the heart of these areas is the need for appropriate, fair, valid, reliable, and useful testing and assessment practices and instruments (see recent considerations of these issues in Fulcher & Harding, 2022). This Special Issue in honour of Professor Glenn Fulcher is a tribute to his work at the heart of these areas. The Special Issue describes this work holistically and across different domains of focus.

show abstract

“…ECD has been successfully used in several research studies that develop and investigate innovative test tasks (e.g., Banerjee, 2019;Choi, 2018;Lee, 2015;Park, 2015). In this dissertation study, an ECD framework was used to provide a solid foundation for test design and development based on Mislevy et al (2003).…”

Section: Evidence-centered Designmentioning

confidence: 99%

Development and validation of spoken dialog system-based oral communication tasks in an ESP context

Karatay

View full text Add to dashboard Cite

show abstract

What interpretations can we make from scores on graphic-prompt writing (GPW) tasks? An argument-based approach to test validation

Cited by 4 publications

References 19 publications

Validity of score interpretations on an online English placement writing test

Validity of score interpretations on an online English placement writing test

lee.mccallum@coventry.ac.uk Language Teaching Research Quarterly 2022, Vol. 29, 1–5 Introduction to the Special Issue in Honour of Professor Glenn Fulcher

Development and validation of spoken dialog system-based oral communication tasks in an ESP context

Contact Info

Product

Resources

About