Information Literacy (IL) Intervention Workshop has Positive, but Limited, Effects on Undergraduate Students’ IL Skills

Abstract

A Review of:
 Gross, M. & Latham, D. (2013). Addressing below proficient information literacy skills: Evaluating the efficacy of an evidence-based educational intervention. Library & Information Science Research, 35(3), 181-190.
 
 Objective – To evaluate the impact of an educational intervention workshop on students’ information literacy (IL) skills and self-perception of their own IL knowledge.
 
 Design – Quasi-experimental design with control groups and semi-structured interviews. 
 
 Setting – Two community colleges in the United States of America, one in a rural setting and one in an urban setting.
 
 Subjects – Ninety-two students enrolled in an entry-level English course, who scored below proficiency (65%) on the Information Literacy Test (ILT). 
 
 Methods – One hundred students from each college took the pre-session ILT and an IL self-assessment survey at the beginning of the Spring 2011 semester. The ILT used was developed and validated by James Madison University (Wise, Cameron, Yang, & Davis, n.d.) and measures understanding of all the Association of College and Research Libraries (ACRL) Information Literacy Competency Standards (ACRL, 2000, pp. 2-3) except Standard 4. For motivation, students each received $20 for their efforts and were told those who scored in the top 15% would enter a draw to win one of two additional prizes of $50. Those who scored below the ILT proficiency level of 65% were invited to participate in the quasi-experiment.
 
 Forty-nine participants were assigned to the workshop group and 43 to the control group. The two groups were comparable in demographic characteristics, prior IL learning, and ILT scores. Those in the workshop group were ask to attend one of five workshops designed around the Analyze, Search, Evaluate (ASE) process model for IL interventions (Gross, Armstrong, & Latham, 2012). The workshops were offered on both campuses and taught by the same instruction librarian.
 
 The workshop participants completed questionnaires, which included a second ILT, self-assessment, and ASE-based questions, before and after the IL workshops. Each workshop participant received $30. The control group participants took the same post-session questionnaire after the workshops were completed and received $20. The same $50 incentive was offered to both groups. Two weeks after the workshops, semi-structured individual interviews were conducted with 30 participants to analyze their learning experiences.
 
 Results – Participants’ self-assessment of IL skills showed significant downgrading after they took the ILT for the first time. This downward calibration held true for both the control (t (41) = 4.077, p < 0.004) and the workshop (t (45) = 4.149, p < 0.000) groups. Subsequent self-ratings from the control group showed this downward recalibration of self-assessment was sustained over time. 
 
 For participants in the workshop group, their average self-rating of IL ability rose from a pre-ASE workshop rating of 2.79 out of a maximum score of 5, to a post-workshop rating of 3.83. However, the same participants’ post-workshop ILT scores did not show any significant improvement. Attending the ASE workshop did not help participants to achieve the “proficient” IL skill level (an ILT score of 65% or higher).
 
 Nonetheless, the workshop group’s performance on the ASE focused questions, also administered pre- and post-session, indicated that participants did gain some IL skills during the workshop. On the ASE questions, which had a maximum score of 25 points, the workshop group’s average score increased from 10.62, pre-session to 13.40, post-session, while the control group had an average score of 10.91 pre-session and 10.77 post-session.
 
 In the follow-up interviews, most participants reviewed the workshop positively and felt that their peers would benefit from attending. However, the skills participants reported learning primarily focused on the Search stage of the ASE model, such as exact phrase, truncation, and the advanced search options in Google. 
 
 Conclusion – This quasi-experiment examined the impact of a one-hour ASE model-based workshop on first-year English students with below-proficiency IL skill levels. Self-assessment ratings indicated that workshop attendance increased students’ confidence in their skill level, although this upward recalibration of self-view significantly overestimated participants’ actual skill gain. Pre- and post-test questionnaires indicated that, while students did gain some new IL knowledge, attending the workshop was insufficient to improve their IL skill to the proficient level.

Digital research notebook: a simple tool for reflective learning

“…Despite these positive survey results, the authors began to question the efficacy of both the workshops and one-shot instruction sessions as singular instructional methods. Shen (2014) demonstrates that while self-assessment ratings of students' confidence seem to have an impact, these self-assessments "significantly overestimated participants' actual skill gain". Similarly, the authors' experience is echoed by the library literature indicating that single library instruction sessions simply do not offer adequate time for research and information literacy skill development (Anderson and May, 2010;Mery et al, 2012).…”