Constructing an evidence-base for future CALL design with ‘engineering power’: The need for more basic research and instrumental replication

Abstract

This paper argues that the goal of Computer-Assisted Language Learning (CALL) research should be to construct a reliable evidence-base with 'engineering power' and generality upon which the design of future CALL software and activities can be based. In order to establish such an evidence base for future CALL design, it suggests that CALL research needs to move away from CALL versus non-CALL comparisons, and focus on investigating the differential impact of individual attributes and affordances, that is, specific features of a technology which might have an impact on learning. Further, in order to help researchers find possible explanations for the success or failure of CALL interventions and make appropriate adjustments to their design, it argues that these studies should be conducted within the framework of Second Language Acquisition (SLA) theory and research. Despite this, a recent review of research examining the effectiveness of CALL in primary and secondary English as a Foreign Language (EFL) found that CALL vs. non-CALL comparisons are still common and studies focusing on individual coding elements are rare. Further, few studies make links with SLA and few measure linguistic outcomes using measures developed in the field of SLA. One reason for this may be poor reporting of methods and difficulty in obtaining the instruments used in SLA research. Reporting guidelines and the use of the IRIS database (www.irisdatabse.org) are introduced as possible solutions to these problems.

The impact of mobile application features on children’s language and literacy learning: a systematic review

“…Fantozzi et al, 2018;Levinson & Barron, 2018;Wang et al, 2020). There is room for more nuanced theorising and hypothesis-testing relating to the impact of app features on language learning which can be sourced from existing qualitative work (Handley, 2014(Handley, , 2018. More specifically, studies are needed to examine a wider range of features of applications (see Table 1), a greater variety of language skills (e.g.…”

Replication research in computer-assisted language learning: Replication of Neri et al. (2008) and Satar & Ӧzdener (2008)

“…The current study is an example of a broad comparison of CALL and current (non-CALL) provision. As has long been acknowledged in the field of CALL, the results of such broad comparisons are constrained to the specific software investigated (Pederson 1987; Garrett 1991; Handley 2014). This is because the CALL condition often differs in multiple ways from the non-CALL condition (Handley 2014).…”

“…As has long been acknowledged in the field of CALL, the results of such broad comparisons are constrained to the specific software investigated (Pederson 1987; Garrett 1991; Handley 2014). This is because the CALL condition often differs in multiple ways from the non-CALL condition (Handley 2014). For example, in this study, in addition to whether or not ASR-based feedback on pronunciation quality is provided, the CALL condition differs from the non-CALL condition with respect to the voice in which the pronunciation models are presented, a variable which other research suggests has an impact on the effectiveness of pronunciation training (Probst, Ke & Eskenazi 2002).…”

“…Explanatory power refers to the extent to which studies are able to provide reasons for any observed impact of the attributes and affordances of technology on language learning (Pederson 1987). Knowledge construction refers to the potential to integrate and synthesise the findings of different individual studies into a coherent evidence-base (Handley 2014).…”

“…This paper, however, argues that, in addition to lack of evidence of validity and reliability also observed in the wider field of SLA due to the lack of replication (Porte 2013), the current CALL evidence-base also lacks generalizability (to other CALL software) and explanatory power, and there is an absence of knowledge construction. It is argued that generalizability to other CALL software might be achieved by focusing in on attributes and affordances which transcend multiple technologies (Pederson 1987; Stockwell 2012), where attributes refer to ‘features of the computer which have the potential to support and develop cognitive processing, such as symbol systems, multimedia and random access’, and affordances refer to ‘features of the computer which enable learners to engage in processes that support language learning’ (Handley 2014: 49). Broad CALL versus non-CALL comparisons, however, dominate the literature (Felix 2008; Macaro et al 2012), and consequently there are currently few insights into the impact of attributes and affordances which transcend multiple technologies.…”