Measuring Cognitive Task Demands Using Dual-Task Methodology, Subjective Self-Ratings, and Expert Judgments

Abstract

This study explored the usefulness of dual-task methodology, selfratings, and expert judgments in assessing task-generated cognitive demands as a way to provide validity evidence for manipulations of task complexity. The participants were 96 students and 61 English as a second language (ESL) teachers. The students, 48 English native speakers and 48 ESL speakers, carried out simple and complex versions of three oral tasks-a picture narrative, a map task, and a decision-making task.Half of the students completed the tasks under a dual-task condition. The remaining half performed the tasks under a single-task condition without a secondary task. Participants in the single condition were asked to rate their perceived mental effort and task diffi culty. The ESL teachers provided expert judgments of anticipated mental effort and task diffi culty along with explanations for their ratings via an online questionnaire. As predicted, the more complex task versions were found and judged to pose greater cognitive effort on most measures.

The Effects of Mode and Task Complexity on Second Language Production

“…As in Révész et al. (), we found that speakers’ ratings of the cognitive load increased significantly from the simple task ( M = 5.08, SD = 2.20) to the complex task ( M = 5.87, SD = 1.98), F (1, 38) = 5.56, p = .024, ƞ p 2 = .13. The same was found for writers, whose ratings increased from the simple ( M = 3.64, SD = 1.58) to the complex condition ( M = 5.03, SD = 1.86), F (1, 38) = 19.04, p < .0001, ƞ p 2 = .33.…”

“…In order to obtain an independent measure of the cognitive load imposed by the task, we employed the Affective Questionnaire (Robinson, 2001), which asks participants to self-assess the mental effort experienced during task performance on a 9-point Likert scale. As in Révész et al (2016), we found that speakers' ratings of the cognitive load increased significantly from the simple task (M = 5.08, SD = 2.20) to the complex task (M = 5.87, SD = 1.98), F(1, 38) = 5.56, p = .024, ƞ p 2 = .13. The same was found for writers, whose ratings increased from the simple (M = 3.64, SD = 1.58) to the complex condition (M = 5.03, SD = 1.86), F(1, 38) = 19.04, p < .0001, ƞ p 2 = .33.…”

“…() were restricted solely to oral performance, their findings were relevant to our study given that a considerable number of the participants in Révész et al. () had the same L1 and L2 learning background as the participants in our study.…”

“…The possibility of using the same task in the two modes was particularly relevant because of the desirability of comparing equivalent communicative tasks in order to be able to isolate mode-related idiosyncrasies in performance (Chafe & Danielewicz, 1987;Tannen, 1982). Also, by employing dual-task methodology, self-ratings, and expert judgments, Révész, Michel, and Gilabert (2016) showed that the complex version of the Fire Chief Task indeed imposed a higher cognitive load, compared to the simple version of this task. Although the results in Révész et al (2016) were restricted solely to oral performance, their findings were relevant to our study given that a considerable number of the participants in Révész et al (2016) had the same L1 and L2 learning background as the participants in our study.…”

Linguistic Dimensions of L2 Accentedness and Comprehensibility Vary Across Speaking Tasks

“…While no tasks. Speakers' perception of the TOEFL task as the most difficult aligned with our view that it would be the most cognitively challenging, similar to the findings in Révész, Michel, and Gilabert (2015), who studied learner versus expert judgments of task complexity. Thus, the three tasks likely represented different task demands, with the picture and IELTS tasks being less demanding than the TOEFL task.…”