Evaluating a web-based video corpus through an analysis of user interactions

Abstract

As shown by several studies, successful integration of technology in language learning requires a holistic approach in order to scientifically understand what learners do when working with webbased technology (cf. Raby, 2007). Additionally, a growing body of research in computer assisted language learning (CALL) evaluation, design and development, has indicated that analysis of learners' behaviours is an essential element to implementing high-quality technology (e.g., Chapelle, 2001;Levy & Stockwell, 2006). Hence, carefully evaluating the effectiveness of CALL by collecting empirical data on user interactions while focusing on the process of learning is integral to a holistic understanding of students' behaviours (e.g. Felix, 2005;He´mard, 2006). This article examines a design-based research that seeks to analyse and understand the dynamics of user interactions with a specific web-based CALL tool in the context of a French as a second language (FSL) course. To this end, we present a sample of results based on an analysis of specific tasks carried out with this CALL tool that is designed in part to encourage students' integration of critical and electronic literacies. By way of conclusion, we identify the steps that are necessary to enhance this particular CALL system and help users better achieve their learning goals. In particular, we explain the process of recycling our results in the next design phase of the CALL tool in a continuous improvement effort.

Gamifying French Language Learning: A Case Study Examining a Quest-based, Augmented Reality Mobile Learning-tool

“…A design-based research, specifically the Analysis-Design-DevelopmentImplementation-Evaluation (ADDIE) model was employed (Strickland, 2006;Colpaert, 2006). As Caws (2013) states: "ADDIE is an instructional system design (ISD) that is particularly well suited to guide developers in the creation and evaluation of language software or other language-related computer systems" (89). Designbased research is an iterative process, and each stage of the ADDIE model produces output that the researcher employs as input for the next stage (Colpaert, 2006, 115).…”

Modes of Annotation in the Video-Based Corpus FrancoToile: Developing a Design Method

“…We analyze and recycle results of the evaluation into the development of the system. As explained in earlier papers (Caws 2009;Caws 2013;Caws and Hamel 2010), each implementation of the system into a learning environment functions as a case study with the goal of promoting students' development of cultural literacies. Part of the data collected during our interventions are of a qualitative nature and come from notes taken by users during their exploration of the system, transcripts of focus groups interviews, and comments made in online surveys.…”

“…As explained earlier, the development of our system-based in part on the ADDIE model-includes a series of evaluations based on several implementations of the tool within the specific educational setting of language courses at the University of Victoria. These case studies include formative and summative evaluations in the form of online surveys, screen captures of interactions, note taking and focus group interviews (Caws 2009;Caws 2013). Data collected are analyzed using statistical measures for quantitative data and discourse analysis for qualitative data.…”

A systematic review of the uses and spread of corpora and data-driven learning in CALL research during 2011–2015

“…ReCALL (11.4%) and CALL (8.3%) published the highest number papers addressing DDL/corpora in language learning during the aforementioned period, although the former devoted a special issue to DDL in 2014 which included eight papers, thus explaining the peak that year. Two ReCALL papers were not retained for further analysis: the first did not research language corpora (Caws, 2013), whereas the second used corpora as a research tool instead of a learning or teaching resource . System (0.9%), LLT (2.5%) and CALICO Journal (5.3%) featured the lowest numbers of published research in our area of interest.…”