Interactional Features of L2 Pragmatic Interaction in Role‐Play Speaking Assessment

Abstract

This study explicates the nature of second language (L2) pragmatic interaction focusing on the quantitative function of interactional features. A relationship between the fine‐grained interactional features elicited from learners’ role‐play performances at varying levels and trained raters’ scores was investigated. The corpus of 102 learners’ role‐play performances was transcribed turn by turn and analyzed qualitatively to identify recurrent interactional phenomena, which formed the basis of data‐driven coding schemes that are grounded in conversation analysis. Three levels of interactional features—length of interaction, engaging with interaction, and sequential organizations—were coded. The coded data were extracted using the CLAN program and analyzed quantitatively to examine what interactional features distinguished among varying performance levels. The linear regression analysis indicated that the combination of nine significant interactional features accounted for 74.7% of the variance in the raters’ scores. The discriminant function analysis revealed that the interactional features elicited from the role‐plays that involve formal pragmatic functions were important variables in distinguishing among the performance levels. This study confirms that the interactional features deserve a prominent place in defining L2 pragmatic interaction. The author discusses practical implications for teaching and assessing interactive pragmatic performance, as well as methodological issues underlying quantifying spoken interaction.

An Argument-Based Validation of an Asynchronous Written Interaction Task

“…The scheme for coding interactional features was developed both deductively and inductively. The initial coding scheme was informed by previous research on interactional competence ( Youn, 2020 ; Nakatsuhara et al, 2021 ) and by the inductive analysis of five test-takers’ responses. In the second stage, two coders, both of whom are PhD candidates in the field of language testing, jointly completed the coding of one test-taker’s performance and revised the coding scheme accordingly.…”

“…Previous research focuses mainly on oral interaction (e.g., Gan, 2010 ; Nakatsuhara et al, 2021 ; Zhang and Jin, 2021 ), whereas written interaction has received less attention ( Abe and Roever, 2019 ; Abe, 2021 ). In addition, scholars have mainly addressed synchronous interaction tasks, such as oral interviews and paired discussion tasks (e.g., Galaczi, 2014 ; Youn, 2020 ), while research on asynchronous written interaction tasks remains rather limited. As existing research findings on synchronous interaction may not necessarily be generalizable to asynchronous interaction, further investigation is needed to attain a more comprehensive understanding of interactional competence.…”

“…Test-takers’ performances and rater’s feedback are the two major perspectives taken by research concerning interactional competence ( Borger, 2019 ). Research focusing on test-taker performances summarizes the interactional features exhibited by test-takers in interaction tasks (e.g., Galaczi, 2008 ; Youn, 2020 ), while research based on rater’s feedback analyzes verbal or written rating reports to explore the interactional features that are salient to raters (e.g., Ducasse and Brown, 2009 ; May, 2009 , 2011 ). These two lines of research have contributed to the operationalization of the interactional competence construct in assessments and provided an analytical framework for interactional competence.…”

“… Galaczi and Taylor (2018) proposed that interactional features include turn management, topic management, interactive listening, breakdown repair and non-verbal features. To capture test-takers’ interaction in role-playing tasks, Youn (2020) coded test performances with three levels of interactional features (length of interaction, engagement with interaction and sequential organization), and found that interactional features exhibit differences across task types. For example, in oral interview tasks, test-takers mainly display the ability to answer questions instead of the capacity to initiate dialogues and change topics, while role-playing tasks can be used to assess their interactional competence with respect to conveying requests or refusals ( Youn, 2020 ).…”

“…To capture test-takers’ interaction in role-playing tasks, Youn (2020) coded test performances with three levels of interactional features (length of interaction, engagement with interaction and sequential organization), and found that interactional features exhibit differences across task types. For example, in oral interview tasks, test-takers mainly display the ability to answer questions instead of the capacity to initiate dialogues and change topics, while role-playing tasks can be used to assess their interactional competence with respect to conveying requests or refusals ( Youn, 2020 ). Although participants in asynchronous written interaction tasks cannot engage in the co-construction of synchronous dialogues through turn-taking, they may still exhibit certain interactional features, such as responding to each other, asking questions or making suggestions.…”