Learning in Simulations: Examining the Effectiveness of Information Literacy Instruction Using Middle School Students’ Portfolio Products

Abstract

Objective –This study compared the effectiveness of simulation-based and didactic instructional approaches in improving students’ understanding of information literacy (IL) concepts and practices. 
 
 Methods – The instructional approaches were implemented with two groups of middle school students (i.e., seventh and eighth grades) over a 4-week period. During the implementation period, all students were required to maintain a portfolio of their work. The portfolios were designed to capture students’ actions as they engaged in a common set of information-based problems. The contents of the portfolios were analyzed to examine the research questions that guided the study. Contingency tables demonstrated observed patterns of difference from week 1 to week 4. Chi-square analysis helped to determine whether a significant relationship existed between instructional approach and shifts in IL proficiency levels at the .05 level. 
 
 Results – There was a significant relationship between the simulation-based approach and increases in students’ ability to 1) recognize the need for information, 2) formulate specific questions that would help in finding needed information, 3) identify a range of information sources for meeting needs, 4) explain successful strategies for accessing needed information, 5) judge the accuracy, relevance and completeness of sources and 6) analyze information from a variety of sources to determine its applicability to a specific problem. Four major distinctions are believed to have caused the students within the simulated instructional environment to experience more proficiency level shifts: situated practice, authenticity, community of practice and an expanded landscape of resources. 
 
 Conclusion – The results of this study suggest that simulation-based instructional approaches have the potential to augment IL learning. The technology-based approaches may provide powerful learning environments (virtual worlds) that allow students to engage in the activities and practice of information specialists, instead of simply learning the facts associated with the discipline.

Game-Based Assessment of Students’ Digital Literacy Using Evidence-Centered Game Design

“…Although several studies have introduced situational task assessment and portfolio assessment as alternative tools for process evaluation in recent years, the process data collected by the situational task assessment is of single type and coarse granularity, which is relatively inadequate in providing comprehensive evidence reflecting students' digital literacy [20,21]. Establishing scientific and objective evaluation standards for portfolio assessment is challenging; it requires teachers to invest a great deal of time and effort, which limits the application of portfolio assessment in evaluating students' digital literacy [22,23]. Therefore, these two assessment methods could not accurately evaluate students' digital literacy [24].…”

How real is real enough? Participant feedback on a behavioral simulation used for information-seeking behavior research

“…This finding aligns with Walsh’s (2009) article reviewing assessment methods for information literacy that feature two articles employing simulation, one paper-based and one online. The latter was an online simulation-based information literacy instructional approach used to guide middle school students through the information-seeking process (Newell, 2010). An additional search of the literature revealed two more studies that explicitly used online simulation.…”

The Changing Nature of Evidence for EBLIP

“…This may be about the "best" type of evidence for a particular question or it may be the "best" type of evidence that can be collected by a particular library at a particular time. For example, EBLIP has published systematic reviews (Koufogiannakis & Weibe, 2006), correlational studies (Eng & Stadler, 2015), quantitative analysis (Newell, 2010), and qualitative studies (Rankin, 2012). Over time as a profession we can look at our evidence base and seek to improve it, but the evidence needs to fit the question and the context and, in the meantime, we need to use the best evidence we can find to help professional decision making.…”