Qualitative Analysis of Faculty Opinions on and Perceptions of Research Impact Metrics

Abstract

We present a qualitative analysis of the results of a survey of faculty and researchers at a large Midwestern R1 university around their understanding of and attitudes toward scholarly metrics. The survey included opportunities for participants to provide freetext responses regarding their use of metrics and concerns they have about the use of metrics for assessment. Participants indicated they understand metrics and use them in a variety of ways, but they have concerns about administrators' potentially inappropriate use of metrics in assessment. Participants expressed a desire to be involved in decision making around the use of metrics in evaluation processes. With the end goal of improving our library's research impact-related services to better support faculty and researchers across campus, this exploratory qualitative analysis offers a more nuanced understanding of the current landscape of opinion around research impact metrics. To develop tools and services that actually address faculty and researcher needs, librarians must develop a comprehensive understanding of their interests and concerns around metrics.

Conjoint analysis of researchers' hidden preferences for bibliometrics, altmetrics, and usage metrics

“…As potential reasons for altmetrics' low popularity, Aung et al (2019) mention academics' insecurities regarding altmetrics' added value, missing encouragement of altmetrics‐ and social media usage from institutions, and privacy concerns, among others. The reoccurring finding of a comparatively low familiarity with altmetrics is in line with multiple other surveys across academic librarians (Miles, Konkiel, & Sutton, 2018) and faculty (Bakker et al, 2019; DeSanto & Nichols, 2017).…”

“…In their follow‐up study, Ma and Ladisch (2019) revealed a discrepancy between interviewed researchers' attitudes toward metrics in practice and in principle: while in principle, most researchers would not trust metrics as objective indicators for quality, in practice they do actively use them for personal or administrative purposes, thereby relying on them as supposedly objective measures. A similar discrepancy was found in a recent qualitative survey across faculty, instructors, and researchers at the University of Minnesota about their attitudes toward research metrics (Bakker, Cooper, Langham‐Putrow, & McBurney, 2020). The participants stated several use cases metrics would fulfill for them, for example, in information‐seeking activities, as a means of self‐assessment, or in the assessment of other individuals.…”

“…The rank order bibliometrics > usage metrics > altmetrics also coincides with findings by Miles et al (2018) about academic librarians' familiarity with different types of research impact indicators. It stands to reason that also for researchers their reluctance to use web‐based indicators can at least in part be explained by their lesser familiarity with them compared to citation‐based metrics (Aung et al, 2019; Bakker et al, 2019; DeSanto & Nichols, 2017; Lemke et al, 2019). Also, citation‐based indicators are the type of indicator that (still) counts the most in the academic reward system.…”

Retracted publications in pharmacy systematic reviews

“…When assessing scientific literature, researchers may use metrics such as the JIF as an indicator of journal quality or rigor [ 41 ]. Our investigation found that both the retracted publications and systematic reviews citing the publication after the retraction was issued were more likely to be in JIF Q1.…”

Could the altmetrics wave bring a flood of confusion for anatomists?

“…While the extant literature often discusses the “usefulness” of altmetrics, many proposed uses are merely hypothetical and rarely demonstrate how to practically apply altmetrics in academic settings. More so, despite the growing body of literature on altmetrics, studies show that academics have little familiarity with altmetrics, and when they do, they typically do not find them valuable for promotion and tenure (P&T) 22,23 …”