Stop chasing unicorns

Abstract

Purpose The purpose of this paper is to discuss the importance of selecting “well-matched” independent and dependent variables in quantitative research to maximize the possibility of detecting impact of library services on indicators of student success. The paper introduces the concept of sensitivity, which is the extent to which a measure will detect change in the thing being measured. Design/methodology/approach To make the case, the authors use the impact of amount of library instruction received on Grade Point Average (GPA) as an example, explaining a correlational research study at their institution. However, the emphasis of the paper is on the conceptual importance of sensitivity in variable selection in quantitative studies of all types. Findings After finding no statistically significant relationship between the amount of library instruction received and GPA, the authors determined that GPA was not a sensitive enough variable to detect the impact of a few class sessions taught by a librarian throughout students entire undergraduate career. Based on the findings and the literature, the authors conclude that the practice of selecting “insensitive” dependent variables that are unlikely to detect impact of the independent variable is a common practice in the library assessment literature. Originality/value In an era where bigger is better when it comes to demonstrating impact of library services, this paper argues that libraries sometimes diminish their ability to illustrate their contributions to student success by choosing large scale indicators of student success as independent variables which fail to detect the impact of library services.

Impact of academic libraries on grade point average (GPA): a review

“…Several research studies found a positive correlation exists between library instruction and GPA (Vance et al , 2012; Wong and Cmor, 2011; Gaha et al , 2018; Bowles-Terry, 2012). Gariepy et al (2017) found no statistically significant relationship between minutes of library instruction and GPA because GPA is not a sensitive enough variable to be impacted by minutes of library instruction. Massengale et al (2016) found students who reserved study rooms earned the highest GPAs.…”

“…Cherry et al (2013) and Wong and Webb (2011) used Pearson correlation to measure the strength of a relationship and a two-tailed significance test to determine if the correlations were statistically significant. Gariepy et al (2017) used a Pearson product-moment correlation and independent samples t -test. Thorpe et al (2016) used a one-sample t -test and chi-square test.…”

“…Bowles-Terry (2012) acknowledged there are confounding variables that affect GPA. Cherry et al (2013, p. 395) stated “the library acknowledges that there are other factors to students' success.” Thorpe et al (2016, p. 384) acknowledged the “complexity of the student experience.” Gariepy et al (2017) found that attendance at a workshop might not have enough of an impact on GPA to see a correlation or relationship between the two and concluded that GPA is not a sensitive enough variable to be impacted by minutes of library instruction. Soria et al (2013) limited their study to only first time, first-year undergraduate students in their first semester in an attempt to minimize nonlibrary variables that affect student achievement.…”

“…Vance et al (2012), Gaha et al (2018) and Bowles-Terry (2012) only looked at presumed attendance from enrollment lists for courses in which a librarian provided instruction. Researchers with large data samples looked at class enrollment and not if the students were physically present in class for a library instruction session (Gariepy et al , 2017; Soria et al , 2013). LeMaistre et al (2018) identified the amount of time spent in library databases vs just the number of sessions as a limitation.…”

The Impact of Library Instruction on Undergraduate Student Success: A Four-Year Study

“…The authors acknowledge, however, that library research using GPA and retention as variables, if done well, has the potential to provide evidence to library or university administration of the library's value. 26 The authors of one of the earliest library studies to attempt to measure effects of library instruction on student success variables-Selegean, Thomas, and Richman, in 1983-voice caution that library instruction efforts must be evaluated by variables that are more sensitive than the large-scale indicators. 27 None of these large-scale student success correlation studies can claim causality of library instruction or library use to better student outcomes, and they certainly do not provide feedback/evidence for fine-tuning instructor performance; however, they have the potential to provide valuable talking points that libraries can use to advocate.…”