Towards Impact Measurement: An Overview of Approaches for Assessing the Impact of Academic Literacy Abilities

Abstract

A variety of academic literacy interventions are used at higher education institutions to address the low level of academic literacy with which many students enter these institutions. Considering the increasingly resource-scarce higher education environment, it is becoming crucial for those who are responsible for such interventions to provide evidence of their impact on student success. The aim of the current study is to provide a broad overview and critique of studies conducted thus far that attempt to assess the impact of various academic literacy interventions. This study proceeds by identifying instruments that are commonly used when assessing the impact of these interventions. From the literature surveyed, it would seem that there are two broad aspects that are considered when evaluating impact, namely students' improved academic literacy levels between the onset and the completion of the course, and the extent to which these acquired academic literacy abilities are transferred to students' other subjects. The next step in this research project will be to propose a comprehensive evaluation design that could be used by a range of academic literacy interventions.

Impact measurement: towards creating a flexible evaluation design for academic literacy interventions

“…As discussed in Fouché (2015), even though some attempts to assess the impact of academic literacy interventions have been made, the type and variety of research instruments used have rarely been sufficient to validly and reliably determine firstly, whether these interventions have an impact, and secondly, what the degree of this impact might be. Reasons for this include that, typically, too few instruments have been used in most studies conducted thus far to make valid conclusions regarding the impact of modules, that instruments used thus far have been varying to such a degree that it might be difficult for future researchers to decide which to adopt, and that they are often designed for local contexts and are not necessarily as robust as one would like in order to make valid and reliable conclusions, which can be compared to those of similar interventions elsewhere.…”

“…Future articles will empirically verify this conceptual evaluation design, and the results from these articles will further inform the final evaluation design. Section 2 briefly describes the variety of instruments that have been used in previous studies to assess the impact of academic literacy interventions; whereas Fouché (2015) focuses on the studies that have been conducted in the field of impact assessment, section 2 focuses on the instruments that have and can be used, in addition to guidelines that are given in relation to the use of these instruments, and what their respective strengths and limitations are. In section 3, an evaluation design is proposed through which a combination of these instruments could be used to assess various academic literacy interventions.…”

“…A previous article (Fouché 2015) provides an overview of studies that have aimed to evaluate the impact of a range of academic literacy interventions. That article discusses each study by considering its research methodology, results, strengths and weaknesses.…”

“…Based on the findings from the literature, broad suggestions of possible useful instruments for impact evaluation are made. The aim of the current article is to build on the literature surveyed in Fouché (2015), and to propose a conceptual evaluation design that could be used to assess the effectiveness of various academic literacy interventions based on that literature. The findings from Fouché (2015) will not be repeated in the current article -rather, for a thorough overview of instruments that have been used to assess the impact of academic literacy interventions, these two articles should be considered in tandem.…”

“…The aim of the current article is to build on the literature surveyed in Fouché (2015), and to propose a conceptual evaluation design that could be used to assess the effectiveness of various academic literacy interventions based on that literature. The findings from Fouché (2015) will not be repeated in the current article -rather, for a thorough overview of instruments that have been used to assess the impact of academic literacy interventions, these two articles should be considered in tandem. Future articles will empirically verify this conceptual evaluation design, and the results from these articles will further inform the final evaluation design.…”