Structural Equation Modeling Reporting Practices for Language Assessment

Ockey, Gary J.; Choi, Ikkyu

doi:10.1080/15434303.2015.1050101

Cited by 39 publications

(30 citation statements)

References 55 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, the participants were mostly on IELTS preparation courses; the participants in both phases of the present study were those who needed to attend IELTS preparation course. As for sample size, it determines the quality of SEM study (Ockey and Choi 2015); the minimum and maximum sample size for SEM is indicated to be 100 to 150 subjects (Ding et al 1995;Khine 2013) and 400 subjects (Boomsma 1987), respectively, or 5-10 subjects for every item or variable (Bentler and Chou 1987).…”

Section: Participants and Contextmentioning

confidence: 99%

Validity of the listening module of international English language testing system: multiple sources of evidence

2018

View full text Add to dashboard Cite

Background: A critical issue in International English Language Testing System (IELTS) lies in the significance of the validity of IELTS listening comprehension test (hereafter IELTS LCM). However IELTS listening validity has been investigated, it has not been investigated with reference to multiple sources of evidence regarding item internal factors. To bridge this gap, we investigated its construct validity with use of structural equation modelling (SEM) and assessed differential item functioning (DIF) through cognitive diagnostic modelling (CDM) and Mantel Haenszel (MH). Methods: In this study, first, the participants signed a consent form for participation in the study; then, 480 participants were administered a proficiency test designed by the university of Cambridge; next, out of 480 participants, 463 participants were administered a 40-item IELTS LCT developed by the University of Cambridge. Finally, the data were analyzed with use of LISREL for probing the construct validity of the test; also, for detecting the potential DIF items, MH and CDM were used to make the results of DIF related findings more reliable. Results: The results of the first study confirmed an appropriate model fit, so that all four constructs, i.e., gap filling, diagram labelling, multiple choice and short answer on IELTS LCT, had a statistically significant contribution to IELTS LCT. However, construct-related evidence may not lead to the whole validity. This given, the second study examined the DIF items to argue the validity of IELTS LCT: MH detected 15 DIF items and CDM detected at least 6 DIF items and at most 12 DIF items. Conclusions: Due to its international nature and worldwide evaluative contribution, IELTS needs to have approximately (not absolutely) a stable factor structure, so that it should be invariant across populations and various cultures. More naturally, a test highly valid in one context might suffer from some degree of invalidity with some related constructs in another context. This in mind, our perspective in this research is not recommended to be taken as a one-size-fits-all model: Neither generalization nor claim is made based on the present study.

show abstract

Section: Participants and Contextmentioning

confidence: 99%

Validity of the listening module of international English language testing system: multiple sources of evidence

2018

View full text Add to dashboard Cite

show abstract

“…In'nami and Koizumi (2011) refer to the Tucker-Lewis index (TLI), the comparative fitness index (CFI) and the mean square approximation error (RMSEA) as the most useful and relevant indices for studying language learning. In general, TLI and CFI values greater than 0.95, as well as RMSEA values of less than 0.6, are considered as acceptable model criteria (Ockey and Choi 2015). However, this model showed a poor approach: v2 (df) = 1473.46 (166), p \ .001, TLI = .63; CFI = .68; RMSEA = .10.…”

Section: The Proposed Structural Modelmentioning

confidence: 97%

Structural relationship between the motivation to study of the second language (De), stability and the second language knowledge

Isatayeva¹,

Smanov²,

Mutanova³

et al. 2018

View full text Add to dashboard Cite

The aim of the study is to point out the structural relationships between the possession of a second language and the construction of the stability, motivation and demotivation of students of the second language. The study involved 168 students in Ahmet Yassawi International Kazakh-Turkish University. The data were analyzed using factor analysis, correlations and modeling of structural equations (SEM). Seven factors of sustainability (life satisfaction, communicative, communicative effectiveness, self-control, strategic competence, metacognitive adaptation and realistic optimism) were determined in factor analysis, six factors of motivation (it should be in the second language, ideal second language, instrumental motivation, parental support, academic motivation, awareness of importance) and six factors of demotivation (negative perception of English-speaking countries, compulsory EFL education, perception of irrelevance textbooks or problems, low self-esteem, inappropriate learning environment, untrained teachers). The modeling of structural equations confirmed that stability, the motivation of the second language and the demotivation of the second language contribute to an increase in the level of proficiency in the second language. demotivation of the second language was more explanatory for the possession of a second language than the motivation for learning a second language. Stability had both direct and indirect influence on the knowledge of the second language through the demotivation of the second language. The findings indicate that teachers should equip students with coping strategies for learning a second language.

show abstract

“…SEM has been used to examine the factor structure of language ability by testing the fit of models to data. Ability and measurement error are modeled separately so that the relationships between abilities can be more precisely examined while separately estimating the impact of measurement error (see In'nami and Koizumi, 2011;Winke, 2014;Ockey and Choi, 2015, for SEM in an L2 assessment field).…”

Section: Correlations Between Size and Depthmentioning

confidence: 99%

Structural Equation Modeling of Vocabulary Size and Depth Using Conventional and Bayesian Methods

Koizumi

In’nami

2020

Front. Psychol.

View full text Add to dashboard Cite

In classifications of vocabulary knowledge, vocabulary size and depth have often been separately conceptualized (Schmitt, 2014). Although size and depth are known to be substantially correlated, it is not clear whether they are a single construct or two separate components of vocabulary knowledge (Yanagisawa and Webb, 2020). This issue has not been addressed extensively in the literature and can be better examined using structural equation modeling (SEM), with measurement error modeled separately from the construct of interest. The current study reports on conventional and Bayesian SEM approaches (e.g., Muthén and Asparouhov, 2012) to examine the factor structure of the size and depth of second language vocabulary knowledge of Japanese adult learners of English. A total of 255 participants took five vocabulary tests. One test was designed to measure vocabulary size in terms of the number of words known, while the remaining four were designed to measure vocabulary depth in terms of word association, polysemy, and collocation. All tests used a multiple-choice format. The size test was divided into three subtests according to word frequency. Results from conventional and Bayesian SEM show that a correlated two-factor model of size and depth with three and four indicators, respectively, fit better than a single-factor model of size and depth. In the two-factor model, vocabulary size and depth were strongly correlated (r = 0.945 for conventional SEM and 0.943 for Bayesian SEM with cross-loadings), but they were distinct. The implications of these findings are discussed.

show abstract

Structural Equation Modeling Reporting Practices for Language Assessment

Cited by 39 publications

References 55 publications

Validity of the listening module of international English language testing system: multiple sources of evidence

Validity of the listening module of international English language testing system: multiple sources of evidence

Structural relationship between the motivation to study of the second language (De), stability and the second language knowledge

Structural Equation Modeling of Vocabulary Size and Depth Using Conventional and Bayesian Methods

Contact Info

Product

Resources

About