Validity Inferences Under High-Stakes Conditions: A Response From Language Testing

Hill, Kathryn; McNamara, Tim

doi:10.1080/15366367.2015.1016334

Cited by 3 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meanwhile, test reliability result also statistically Journal on English as a Foreign Language, 13(1), 25-55 p-ISSN 2088-1657e-ISSN 2502-6615 shows positive value. It is in accordance with the theoretical literature of test item development that the empirical validity should be well measured to know how closely scores on a test either correspond or correlate with aspects required in the learning process (Hill & McNamara, 2015;Tosuncuoglu, 2018). Apart from empirical validity test, a reliability test should be conducted to support the consistent result of test item validity (Hamavandy, 2014) (Fan & Jin, 2020).…”

Section: Discussionsupporting

confidence: 56%

“I can’t search on Google for answers”: validity evidence of a developed computer-based assessment application

Nimasari

Gestanti²,

Nurfitri³

2022

J. Eng. Foreign. Lang.

View full text Add to dashboard Cite

While learning management systems have been increasingly used as innovative ways to teach students in the post-COVID-19 pandemic, academic dishonesty is still in crisis. The open-access facility has the potential intervention in academic dishonesty. This study reported validity evidence of a developed computer-based assessment application to combat academic dishonesty: 1) measuring the development of high-order-thinking-skill (HOTS) test items; 2) examining the test item's validity integrated into the application; and 3) reporting the program analysis of the anti-cheating application for English for Specific Purposes (ESP) class assessment. This mixed-method study used focus group discussion and reading tests to collect the data. Pearson product-moment and black box analysis were used to measure test-item validity and application. The quantitative result revealed that test items were developed according to authentic journal texts and HOTS question characteristics and were statistically proven valid and reliable. Simplifying the authentic text empirically enabled EFL students to have an easier ESP reading comprehension complexity level. The black box analysis showed that the auto-lock screen feature prevented suspicious cheating access. This study contributed to extending practical and theoretical research on technology-enhanced language learning (TELL) by providing an innovative anti-cheating technology application.

show abstract

Section: Discussionsupporting

confidence: 56%

“I can’t search on Google for answers”: validity evidence of a developed computer-based assessment application

Nimasari

Gestanti²,

Nurfitri³

2022

J. Eng. Foreign. Lang.

View full text Add to dashboard Cite

show abstract

“…Testing, as mentioned above, is one of those areas that has become a cornerstone in language education (Tsang, Katz, & Stack, 2008;Heilig, 2011;Gutiérrez Eugenio & Saville, 2017), considering both its benefits and drawbacks (Bracey, 2009;Fitchett & Heafner, 2010;Howie, 2012;Koretz, 2016). Language testing is currently used to take decisions in student mobility, student's acceptance in bilingual higher education programs and also to assess language programs (Mahon, 2006;Wright & Choi, 2006;Solano-Flores & Li, 2013;Hill & McNamara, 2015). However, although large scale national assessments tend to be more and more common, they also tend to ignore the relationship between the candidates' personal factors and their grades in high stakes exams (Qian, 2009;Gorges & Kandler, 2012;Bai, Hudson, Millwater, & Tones, 2013) mostly because a great disparity may evidence that practice is not adequate or that socioeconomic problems may be creating significant differences within a country (Fitchett & Heafner, 2010;Kim, 2011;Solano-Flores & Li, 2013).…”

Section: Social Influences Of Language Learningmentioning

confidence: 99%

Differences and Similarities between Oral and Written Competence in Spanish Pre-University Students: A Correlational Study

Laborda

Díez-Bedmar²,

Monje

et al. 2018

Onomázein

View full text Add to dashboard Cite

“…Within this view, meaningfulness is intrinsically connected with a test’s uses and its effects on wider concerns such as language policy. Recent interactions between language policy and testing are hoped “to broaden our thinking about measurement [in order] to consider the policy context of assessment” (Hill & McNamara, 2015, p. 42).…”

mentioning

confidence: 99%

Critical discursive approaches to evaluating policy-driven testing: Social impact as a target for validation

Shin

2023

Language Testing

View full text Add to dashboard Cite

This paper addresses the intersection of testing and policy, situating test-driven impact and validation within the context of policy-led educational reform in Korea. I will briefly review the existing validation models. Then, arguing for an expansion of the conventional conceptualization of consequential validity research, I use Fairclough’s dialectic–relational approach in critical discourse analysis (CDA), positioned in critical and poststructuralist research tradition, to evaluate social realities, such as intended and actual impact of policy-led testing, I take, as an example, the context of the development of the National English Ability Test (NEAT) in Korea, which had been used as a means of implementing government policies. Combining Messick’s validity framework for consequential evidence, Bachman and Palmer’s argument-based approach to validation (assessment use argument, AUA), and Fairclough’s dialectic–relational approach, I will illustrate how the impact of policy-led testing is performed and interpreted as a sociopolitical and discursive phenomenon, constituted and enacted in and through “discourse.” By revisiting the previous Faircloughian research works on NEAT’s impact, I postulate that the discourses arguing for and against social impact acquire their meanings from dialectical standpoints.

show abstract

Validity Inferences Under High-Stakes Conditions: A Response From Language Testing

Cited by 3 publications

References 12 publications

“I can’t search on Google for answers”: validity evidence of a developed computer-based assessment application

“I can’t search on Google for answers”: validity evidence of a developed computer-based assessment application

Differences and Similarities between Oral and Written Competence in Spanish Pre-University Students: A Correlational Study

Critical discursive approaches to evaluating policy-driven testing: Social impact as a target for validation

Contact Info

Product

Resources

About