Chapter 7. Independently measuring cognitive complexity in task design for interlanguage pragmatics development

Gilabert, Roger; Barón, Júlia

doi:10.1075/tblt.10.07gil

Cited by 10 publications

(6 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Third, the role-plays prior and subsequent to the viewing were rated by three English native speakers (one North American, and two British native speakers) who were English language teachers. The raters were trained on how to assess the role-plays by using a pragmatics rating scale adapted from Gilabert & Barón (2018), in which they had to rate pragmatic appropriateness on a Likert-scale ranging from 0 to 6. A ‘0’ was given when the role-play was considered to be pragmatically incorrect or inappropriate.…”

Section: Methodsmentioning

confidence: 99%

‘May I do something for you?’: The effects of audio-visual material (captioned and non-captioned) on EFL pragmatic learning

Barón

Celaya

2022

Language Teaching Research

Self Cite

View full text Add to dashboard Cite

The present study deals with the effect of audio-visual material for second language (L2) pragmatic learning in the foreign language classroom. More specifically, it analyzes whether being exposed to captioned and non-captioned input in an experimental condition entailing no instruction on pragmatics might have any influence on the learners’ pragmatic performance. To this aim, two intact classes ( N = 31) of learners of English as a foreign language (EFL) at a B1 level were exposed to videos with captions and without captions, respectively. Before and after watching the videos, all the students were asked to carry out a role-play task with situations like those in the videos. A mixed-methods approach was used to analyze the learners’ performance in terms of types and number of strategies to perform speech acts (quantitative) and in terms of pragmatic appropriateness (qualitative). Findings show that both groups used more polite strategies after watching the videos, regardless of the captioned/non-captioned condition, which seems to confirm the contribution of audio-visual material for the learning of the L2 pragmatics in an incidental way. Concerning pragmatic appropriateness, we found that learners in the captioned condition produced more pragmalinguistically appropriate role-plays than learners in the non-captioned condition, thus suggesting a positive effect of captioned material on the learning of the L2 pragmatics. Such results are discussed in relation to the few previous similar studies in the field.

show abstract

Section: Methodsmentioning

confidence: 99%

‘May I do something for you?’: The effects of audio-visual material (captioned and non-captioned) on EFL pragmatic learning

Barón

Celaya

2022

Language Teaching Research

Self Cite

View full text Add to dashboard Cite

show abstract

“…Self‐report, protocol questions might include, “How are you going to go about solving this problem?” (Baxter, Elder, & Glaser, 1994, 1996). Other examples include student perception of difficulty (Revesz, 2014a), time spent (Baralt et al., 2014; Lee, 2019; Sasayama, 2016), and expert judgments of task difficulty and mental effort (Gilabert & Baron, 2018; Revesz et al., 2016; Revesz, 2014b). However, there are also useful taxonomies that define and explicate the different levels of task complexity and knowledge such as Bloom's taxonomy and Webb's Depth of Knowledge (DOK).…”

Section: Cognitive Complexitymentioning

confidence: 99%

“…(Baxter, Elder, & Glaser, 1994, 1996. Other examples include student perception of difficulty (Revesz, 2014a), time spent (Baralt et al, 2014;Lee, 2019;Sasayama, 2016), and expert judgments of task difficulty and mental effort (Gilabert & Baron, 2018;Revesz et al, 2016;Revesz, 2014b).…”

Section: Cog Nitive Comple Xit Ymentioning

confidence: 99%

Transparency of cognitive complexity in performance assessments: A validity study

Hayes

Demeter

Morris

et al. 2021

J of Competency-Based Ed

View full text Add to dashboard Cite

Performance assessments (PAs) offer a more authentic measure of higher order skills, which is ideal for competency‐based education (CBE) especially for students already in the workplace and striving to advance their careers. The goal of the current study was to examine the validity of undergraduate PA score interpretation in the college of IT at a CBE online, higher education institute by evaluating (a) the transparency of cognitive complexity or demands of the task as communicated through the task prompt versus expected cognitive complexity based on its associated rubric aspect and (b) the impact of cognitive complexity on task difficulty. We found that there is a discrepancy in the communicated versus expected cognitive complexity of PA tasks (i.e., prompt vs. rubric) where rubric complexity is higher, on average, than task prompt complexity. This discrepancy negatively impacts reliability but does not affect the difficulty of PA tasks. Moreover, the cognitive complexity of both the task prompt and the rubric aspect significantly impacts the difficulty of PA tasks based on Bloom's taxonomy but not Webb's DOK, and this effect is slightly stronger for the rubric aspect than the task prompt. Discussion centers on how these findings can be used to better inform and improve PA task writing and review procedures for assessment developers as well as customize PAs (their difficulty levels) to different course levels or individual students to improve learning.

show abstract

“…TBLA aims to investigate how performance may be assessed through linguistic data collected by means of real-world instructional tasks in which the primary focus is on meaning (Ellis & Shintani, 2014). In this type of research various rating instruments have been employed, as shown by the two studies presented below, one by Kuiken and Vedder (2014, 2017, 2018 on the assessment of functional adequacy, the other by Gilabert and Barón (2018) on measurement of cognitive complexity and pragmatic competence. Kuiken and Vedder (2014, 2017, 2018 developed a six-point Likert rating scale for the assessment of functional adequacy.…”

Section: Recommendations For Practicementioning

confidence: 99%

“…Finally, the findings show the necessity of rater training and monitoring of raters over time (see Pill & Smart,Chapter 13,this volume). Gilabert and Barón (2018) investigated the relationship between task-complexity and pragmatic competence in the L2 writing of intermediate learners of English as a Foreign Language (EFL). The participants had to write a response to four e-mail messages, at different levels of task complexity.…”

Section: Assessment Of Functional Adequacymentioning

confidence: 99%

Scoring Approaches

Kuiken¹,

Vedder²

2020

The Routledge Handbook of Second Language Acquisition and Language Testing

View full text Add to dashboard Cite

Rating scale/rubric: "[A] series of ascending bands of proficiency" against which language performances are judged. "It may cover the whole conceptual range of learner proficiency, or it may just cover the range of proficiency relevant to the sector or institution concerned" (Council of Europe, 2001, p. 40).Rating criteria: Aspects of the construct in terms of which a test-taker's performance will be evaluated, e.g., task achievement, linguistic accuracy, organization, fluency, etc.Scoring band: A level in a rating scale which describes the specific extent to which a language learner can do something in the language and what the linguistic features are of their language use. A rating scale comprises multiple bands.Descriptor: A statement that stipulates the characteristics of a language performance at a particular band level. A scoring band comprises one or more descriptors per rating criterion.

show abstract

Chapter 7. Independently measuring cognitive complexity in task design for interlanguage pragmatics development

Cited by 10 publications

References 47 publications

‘May I do something for you?’: The effects of audio-visual material (captioned and non-captioned) on EFL pragmatic learning

‘May I do something for you?’: The effects of audio-visual material (captioned and non-captioned) on EFL pragmatic learning

Transparency of cognitive complexity in performance assessments: A validity study

Scoring Approaches

Contact Info

Product

Resources

About