Applying learning strategy questionnaires: problems and possibilities

Abstract

This article discusses measuring learning strategies by means of questionnaires. In 'multi-method' research, in which think-aloud measures are compared with questionnaires, low or moderate correlations are found. A conclusion often drawn is that learners are not able to verbally report on their learning activities. Alternative explanations concern two other possibilities: first, that different learning strategies may be measured by the two methods; second, that the measuring methods may be aimed at different learning tasks. Keeping these prerequisites in mind, we constructed a task-specific questionnaire directly based on a taxonomy for coding think-aloud protocols in text studying. We found a higher correlation (r=.51) between the questionnaire and think-aloud protocols than is regularly reported. A case-study, in which four students answered the questionnaire while thinking aloud, led to new insights into why a questionnaire may lead to somewhat different ratings of activities than the think-aloud method. Based on these results, task-specific questionnaires may be improved. Our studies involved a fair comparison between a questionnaire and think-aloud protocols. We cautiously conclude that if task-specific questionnaires are meticulously constructed and examined in new ways, they might become reasonably adequate alternatives for the labor-intensive think-aloud method in measuring learners' learning strategies.Keywords Learning strategy questionnaires . Think-aloud method IntroductionThe ongoing focus on metacognition and learning strategies in general is resulting in various measuring methods and instruments (Veenman et al. 2006). Both in educational practice and in research different kinds of measuring methods are used to record learning strategies. Obviously, different instruments may lead to differences in data and conclusions. This article will principally address the use of questionnaires in measuring learning Metacognition Learning (2011) strategies. First, we will present some distinctions in learning strategies and in measuring methods. We consider the possibilities in task-specific measuring, then we concentrate on to the discussion about validity issues concerning questionnaires by presenting multi-method studies in which questionnaires are compared with other instruments. Finally, we discuss our empirical studies in constructing a task-specific learning strategy questionnaire. Different learning strategiesDefinitions of learning strategies vary. Different terms (e.g., methods, techniques, or skills) are used to describe the activities that are considered to constitute a learning strategy. Learning strategies may be defined as certain combinations of goal-oriented learning activities used by learners to improve learning (cf. Van Hout-Wolters 1992;Rachal et al. 2007). A learner can use all kinds of strategies to improve learning. These can be found in a multitude of divisions and overviews of learning strategies (see e.g., Alexander 2006;Pintrich 2004;Weinstein and Hume 1998;Winne and Hadwin...

Towards efficient measurement of metacognition in mathematical problem solving

“…As discussed in the theoretical framework, this problem is likely to be caused by memory distortions as well as by variation in interpretation of the questions (McNamara 2011;Veenman 2011b). It could be that such issues can be addressed by fitting the formulation of the items more closely to the learning task (Schellings 2011). However, in our study we found that fitting the formulation of the questions to the learning domain (in this case by adding the word mathematics) does not seem to make the statements specific enough.…”

“…Firstly, the fact that self-report questionnaires are collected separate from the learning task means that students have to retrieve earlier processes and performance from their long term memory. Self-report questionnaires thus are susceptible to memory distortion issues (McNamara 2011;Schellings 2011;Veenman 2011b). Secondly, students can differ in their frame of reference as to which situations they have in mind when answering the questions and interpreting the scales (McNamara 2011;Schellings 2011).…”

“…Self-report questionnaires thus are susceptible to memory distortion issues (McNamara 2011;Schellings 2011;Veenman 2011b). Secondly, students can differ in their frame of reference as to which situations they have in mind when answering the questions and interpreting the scales (McNamara 2011;Schellings 2011). Thirdly, the way students answer self-report questionnaires may be biased by triggers in the questions which prompt them to wrongly label their own behavior or by social desirability (Cromley and Azevedo 2011;Veenman 2011a).…”

“…Although sometimes slowing learning down, when executed correctly think-alouds do not impair students' learning performance (Bannert and Mengelkamp 2008;Fox et al 2011). However, besides these positive characteristics, there is a major drawback of the method: Gathering and scoring the data of individual students' think-aloud protocols is a complex and time-consuming process which makes this measure inappropriate for test assistants or teachers who lack experience using the method, and for application in larger samples of students Schellings 2011). Thus, when using this theoretically grounded measure, it tends to conflict with some more practical constrains of time and effort.…”

A Cognitive Approach to L2 Reading Comprehension and the Limits of Description

“…5 That they work is generally agreed upon. See Wilson and Bai (2010). specific, but what cannot be disputed is that there is an overabundance of them (Schellings, 2011). Case in point, Block (1986) touts the success rate of "integrators" versus "non-integrators," Oxford (1990) presents us with a kaleidoscope of learning strategies that she organizes into six different groups: cognitive, metacognitive, memory-related, compensatory, affective, and social.…”