Perceptual Learning Styles and Lessons in Psychometric Weakness

Abstract

In this critical review, I argue that the usefulness of perceptual learning styles constructs within applied linguistics is very limited. Researchers within applied linguistics have neglected to engage with objections to these constructs which date back to the 1970s within general educational research. Problems of poor instrumentation are considerable and predictive power has not sufficiently been demonstrated. It is argued that these constructs present a special case for measurement because they are not easily operationalized through statements on self-report questionnaires. The review discusses implications for practitioners and research, and argues for greater editorial oversight in preventing poor instruments from entering the literature in the future. Some specific recommendations which may assist with such prevention are discussed. These include a more critical approach to the use of Cronbach’s alpha, the use of Confirmatory Factor Analysis (CFA) as one powerful tool to demonstrate unidimensionality, and the avoidance of paraphrased items. 本論文は、応用言語学で用いられる知覚学習スタイルの構成概念が有用性に欠けることを指摘する。一般的な教育学研究の分野で1970年代から批判されてきたこれら構成概念について、応用言語学研究者たちはその対応法の検討を十分には行なっていない。知覚学習スタイルの測定方法には問題が多く、その妥当性も明確になっていない。さらに、アンケートのような自己報告を用いた測定方法では、このような構成概念は適切に定義づけることが難しいことも指摘されている。外国語教育実践と応用言語学研究のためにも、不十分な測定方法が将来的に研究に入り込まないようにする必要があり、そのためには学術誌等の編集者によるさらなるチェック体制の強化が不可欠である。具体的方法として、クロンバックα係数の使用に関する注意喚起や、１次元性を確認するのに有用な確証的因子分析(CFA)の活用を提案する。

JALT Journal

“…Bollen-Stine bootstrapping (Byrne, 2001) was also adopted to overcome the limitations associated with the multivariate nonnormality of the data, and it is important to note the associated positive result from this procedure, which is discussed further below. Nunnally and Bernstein's (1994) criterion of .70 was initially set as the threshold to evaluate the reliability estimates (Cronbach's alpha) for each of the subscales, but this criterion should be adopted critically (Isemonger, 2012). The number of items, which was low for each construct, was also considered in the application of the criterion.…”

“…However, Cortina (1993) and Green, Lissitz, and Muliak (1977) pointed out that alpha is biased by the number of items on a scale, with larger numbers of items producing higher alphas. This property of alpha, and the propensity for some research areas to neglect this when interpreting alpha, has been explained by Isemonger (2012) in a recent and approachable critical review. Only three items comprise each subscale on the CDS II, which is a comparatively low number of items for a subscale and, in fact, close to the minimum for measurement of a latent.…”

“…The paper reports on the adaptation from the domains of sports and psychology to SLA and from the North American college population (the population for which the instrument was originally developed) to the Japanese college population, using the method of Confirmatory Factor Analysis (CFA) as the primary analytical procedure. CFA is the measurement component of Structural Equation Modelling (SEM; Byrne, 2001;Isemonger, 2012) and directly tests the dimensionality of scores produced by an instrument against a measurement model (i.e., the structural validity of the scores); the measurement model is invariably implied by the scoring regime for the instrument. If the measurement model is rejected in a direct test using CFA, then the practitioner should be wary, if not wholly skeptical, about using the instrument for classroom diagnostics as the dimensional properties of scores produced will not cohere with the scoring regime for the instrument.…”