Managing proposal sequences in role-play assessment: Validity evidence of interactional competence across levels

Abstract

This qualitative study reports an investigation of the nature of interactional competence at various levels of achievement in the context of role-play speaking assessment. The focal point of this study is on how examinees jointly accomplish the interactional work involved in proposal sequences in role-play interaction. Based on a conversation analysis of a corpus of role-play interaction, I argue that distinct sequential organizations and interactional features found across examinees’ levels serve as critical validity evidence for assessing interactional competence. Various shift markers and stepwise transitions were present in higher-level examinees when they initiated and shifted actions in role-play interaction. However, lower-level examinees’ opening turns were typically forwarded without establishing a shared understanding relevant to an upcoming action. When the examinees responded to various proposal sequences, coherent and sufficient topic organizations were recurrent in higher-level performances. The examinees, regardless of levels, managed to close the role-play interaction well. I discuss the implications of the demonstrated link between the recurrent interactional features and examinees’ interactional competence for future research into speaking assessment and teaching.

Interactional Features of L2 Pragmatic Interaction in Role‐Play Speaking Assessment

“…Nonetheless, previous research (e.g., Huth, ; Okada, ) indicated that simulated role‐play interaction shares diverse conversational features reflective of real‐life conversation, thereby lending support for meaningful interpretation about interactional competence. Finally, a CA‐based study on learners’ role‐play interaction (Youn, ) provides further evidence that the performance elicited the linguistic and interactional characteristics conforming to the findings in the literature. Taken together, the role‐plays used in this study are deemed to measure the targeted construct.…”

“…Lam’s () CA‐based study on group speaking assessment discourse revealed that speakers’ abilities to respond to a previous speaker were noticeably different depending on their levels. Youn () further supported that CA’s real‐time details contribute to the production of validity evidence in assessing interactive speaking performance. For example, learners’ abilities to propose and shift topics while accomplishing pragmatic actions (e.g., agreement, disagreement) systematically differed depending on their levels.…”

“…The transcribed data were analyzed using CA. The CA findings revealed the recurrent interactional features indicative of varying levels of role‐play performance (see Youn, , for more details). The qualitative findings informed the development of interactionally grounded coding schemes (Step 4), which is further discussed in the next section.…”

“…Similarly, the higher level learners in this study indeed produced more words per turn in formal role‐plays and made more suggestions in order to sustain the interaction regardless of the role‐plays. For the sequential organizations, previous CA‐based research reported that the use of preliminary turns (Al‐Gahtani & Roever, ) and extended topic transition sequences using varied prefacing devices (e.g., discourse markers, topic‐shift markers; Youn, ) were important indicators that distinguished among the role‐play performances. With regard to engagement with the interaction, which was the significant variable in this study, the higher level learners tended to actively engage in the interaction using more frequent suggestions or counter‐suggestions rather than acknowledgment tokens only.…”

“…Research on assessing L2 pragmatic interaction has been increasingly available (e.g., Grabowski, ; Ross & Kasper, ; Youn, ), as have qualitative accounts of what interactional features characterize pragmatic interaction at varying performance levels (e.g., Al‐Gahtani & Roever, , ; Youn, ). Nonetheless, quantitative evidence for what interactional features count as competent L2 pragmatic interaction is still limited.…”