Diagnosing writing ability using China’s Standards of English Language Ability: Application of cognitive diagnosis models

Assessing Students’ Translation Competence: Integrating China’s Standards of English With Cognitive Diagnostic Assessment Approaches

“…Somewhat different from the steps mentioned by Fan and Yan (2020) in diagnosing receptive language skills, the procedures in this study bore a strong resemblance to those in diagnosing writing skills ( Kim, 2010 , 2011 ; Xie, 2017 ; Effatpanah et al, 2019 ; He et al, 2021 ) which included five major stages: identifying the attributes, developing the diagnostic checklist, constructing the Q-matrix, rating the translation responses and selecting the CDM.…”

“…Taking a step forward, quite a few studies have explored the extent to which diagnostic results could help discover the relationships among attributes (e.g., Chen and Chen, 2016a ; Ravand and Robitzsch, 2018 ; Du and Ma, 2021 ) and could differ across different proficiency groups (e.g., Kim, 2010 , 2011 ; Fan and Yan, 2020 ). Among all those studies, a notable phenomenon is that assessment of receptive language skills such as reading and listening (e.g., von Davier, 2008 ; Jang, 2009b ; Lee and Sawaki, 2009b ; Kim, 2015 ; Chen and Chen, 2016b ; Yi, 2017 ; Aryadoust, 2021 ; Dong et al, 2021 ; Toprak and Cakir, 2021 ; Min and He, 2022 ) gained much more attention than that of productive ones such as writing ( Kim, 2010 , 2011 ; Xie, 2017 ; Effatpanah et al, 2019 ; He et al, 2021 ). “One possible reason is that different test methods are used to assess these two types of skills” ( He et al, 2021 , p. 1).…”

“…Among all those studies, a notable phenomenon is that assessment of receptive language skills such as reading and listening (e.g., von Davier, 2008 ; Jang, 2009b ; Lee and Sawaki, 2009b ; Kim, 2015 ; Chen and Chen, 2016b ; Yi, 2017 ; Aryadoust, 2021 ; Dong et al, 2021 ; Toprak and Cakir, 2021 ; Min and He, 2022 ) gained much more attention than that of productive ones such as writing ( Kim, 2010 , 2011 ; Xie, 2017 ; Effatpanah et al, 2019 ; He et al, 2021 ). “One possible reason is that different test methods are used to assess these two types of skills” ( He et al, 2021 , p. 1). For the former, selected-response tests are often used, producing answers that can be scored with 1 (right) or 0 (wrong), which can serve as the response data directly analyzed by CDMs.…”

“…Kim’s study provided methodological references for cognitive diagnostic research conducted with constructed-response tests, but descriptors in the study needed further amelioration in that they were sourced merely from a few teachers’ verbal reports. He et al (2021) made an improvement in this regard by using descriptors selected from CSE. One slight flaw in their study, however, was that only one type of writing activity was involved, which, to some degree, restricted the inferences of test-takers’ writing competence.…”

“…However, no optimal model has been found in writing competence diagnosis due to the limited number of relevant research. Among the very few studies, three models have been found with the best model fit data including RRUM ( Kim, 2010 , 2011 ; Xie, 2017 ), ACDM ( Effatpanah et al, 2019 ), and LLM ( He et al, 2021 ). To the best of our knowledge, it seems that cognitive diagnostic studies on translation competence are not yet available.…”