Jump-Starting Item Parameters for Adaptive Language Tests

McCarthy, Arya D.; Yancey, Kevin P.; LaFlair, Geoff T.; Egbert, Jesse; Liao, Manqian; Settles, Burr

doi:10.18653/v1/2021.emnlp-main.67

Cited by 10 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LLMs and generative AI bring new promise in this area. For example, it has been shown that LLM‐based methods can significantly reduce the number of test takers needed in pilot tests to calibrate the item parameters to the same level of accuracy (McCarthy et al., 2021). This field is undergoing active research, and LLMs and generative AI have the potential to transform this area.…”

Section: Impacts and Implications For Assessmentmentioning

confidence: 99%

Transforming Assessment: The Impacts and Implications of Large Language Models and Generative AI

Hao,

von Davier,

Yaneva

et al. 2024

Educational Measurement

View full text Add to dashboard Cite

The remarkable strides in artificial intelligence (AI), exemplified by ChatGPT, have unveiled a wealth of opportunities and challenges in assessment. Applying cutting‐edge large language models (LLMs) and generative AI to assessment holds great promise in boosting efficiency, mitigating bias, and facilitating customized evaluations. Conversely, these innovations raise significant concerns regarding validity, reliability, transparency, fairness, equity, and test security, necessitating careful thinking when applying them in assessments. In this article, we discuss the impacts and implications of LLMs and generative AI on critical dimensions of assessment with example use cases and call for a community effort to equip assessment professionals with the needed AI literacy to harness the potential effectively.

show abstract

Section: Impacts and Implications For Assessmentmentioning

confidence: 99%

Transforming Assessment: The Impacts and Implications of Large Language Models and Generative AI

Hao,

von Davier,

Yaneva

et al. 2024

Educational Measurement

View full text Add to dashboard Cite

show abstract

“…The linguistic features of these passages have been carefully analyzed to ensure a variety of text types and difficulty levels. In total, more than 150 linguistic features are annotated and accounted for, including features related to parts of speech, verb types, and passage length (see McCarthy et al, 2021, for the complete list).…”

Section: C-testmentioning

confidence: 99%

Duolingo English Test :Technical Manual

Cardwell¹,

Naismith²,

LaFlair³

et al. 2023

View full text Add to dashboard Cite

The Duolingo English Test Technical Manual provides an overview of the design, development, administration, and scoring of the Duolingo English Test. Furthermore, the Technical Manual reports validity, reliability, and fairness evidence, as well as test-taker demographics and the statistical characteristics of the test. This is a living document whose purpose is to provide up-to-date information about the Duolingo English Test, and it is updated on a regular basis.

show abstract

“…Others have identified a range of linguistic features on the word, sentence, and text levels that impact exercise difficulty (e.g. Galasso, 2018;Beinborn et al, 2014;McCarthy et al, 2021;Settles et al, 2020;Brown, 1989). The effect of exercise format parameters such as gap size, deletion pattern and deletion frequency on exercise difficulty varied across studies (Sigott, 1995;Lee et al, 2019;Kamimoto, 1993).…”

Section: Related Workmentioning

confidence: 99%

On the relevance and learner dependence of co-text complexity for exercise difficulty

Heck¹,

Meurers²

2023

Linköping Electronic Conference Proceedings

View full text Add to dashboard Cite

Adaptive exercise sequencing in Intelligent Language Tutoring Systems (ILTS) aims to select exercises for individual learners that match their abilities. For exercises practicing forms in isolation, it may be sufficient for sequencing to consider the form being practiced. But when exercises embed the forms in a sentence or bigger language context, little is known about how the nature of this co-text influences learners in completing the exercises.To fill the gap, based on data from two large field studies conducted with an English ILTS in German secondary schools, we analyze the impact of co-text complexity on learner performance for different exercise types and learners at different proficiency levels. The results show that co-text complexity is an important predictor for a learner's performance on practice exercises, especially for gap filling and Jumbled Sentences exercises, and particularly for learners at higher proficiency levels.

show abstract

Jump-Starting Item Parameters for Adaptive Language Tests

Cited by 10 publications

References 0 publications

Transforming Assessment: The Impacts and Implications of Large Language Models and Generative AI

Transforming Assessment: The Impacts and Implications of Large Language Models and Generative AI

Duolingo English Test :Technical Manual

On the relevance and learner dependence of co-text complexity for exercise difficulty

Contact Info

Product

Resources

About