The present study examines the relative importance of attributes within and across items by applying four cognitive diagnostic assessment models. The current study utilizes the function of the models that can indicate inter-attribute relationships that reflect the response behaviors of examinees to analyze scored test-taker responses to four forms of TOEFL reading and listening comprehension sections. The results are interpreted to determine whether the subskills defined in each subtest contribute equally within an item and across items. The study also discusses whether the empirical results support the claims for compensatory processing of L2 comprehension skills and presents practical implications of the findings. The article concludes with the limitations of the study and suggestions for future research. KeywordsAttributes in second language comprehension tests, cognitive diagnostic models, relative difficulty of L2 subskills, relative importance/contribution of L2 reading or listening skills Research on second language attributes that are assessed through reading and listening comprehension test items has been conducted using a variety of methods. First, to identify the specific subskills measured by L2 reading and listening tests, many studies have employed introspective or retrospective methods such as verbal protocols or interviews Downloaded from 2 Language Testing to elicit learners' reports. A number of studies have also employed statistical and/or psychometrical analysis techniques that focused on surface task characteristics such as item difficulty (Sawaki, Kim, & Gentile, 2009).In relation to this line of studies, another body of research has focused on the hierarchical relationships among L2 receptive subskills in terms of difficulty. These studies have examined whether the existing or suspected hierarchy of attributes could be empirically validated by comparing the hierarchy with actual student performance (e.g., Alderson & Lukmani, 1989) and by investigating the performance differences on the subskills among various levels of learners (Mecartty, 1998).Generalizing these findings to other L2 comprehension tests faces the limitation that most of the test items in these studies were matched to only one subskill. Thus, the relative roles of the attributes are not well known when items are coded for more than one subskill. Considering that many L2 test items are likely to require more than one skill, it seems necessary to scrutinize the relationships among them by analyzing tests at the item level.The present study attempts to fill this research gap by applying psychometric models of cognitive diagnostic assessment to probe the processes of multiple (in this study, two) attributes within an item or across items. As quantitative analysis tools of a measurement procedure, cognitive diagnostic models have dual functions. They primarily estimate the skill mastery profiles of test-takers by classifying students according to their mastered skills and extract detailed information about their strengths and wea...

No abstract

No abstract

No abstract

No abstract