How are we Measuring Up? Evaluating Research Data Services in Academic Libraries

Abstract

INTRODUCTIONIn the years since the emergence of federal funding agency data management and sharing requirements (http://datasharing.sparcopen.org/data), research data services (RDS) have expanded to dozens of academic libraries in the United States. As these services have matured, service providers have begun to assess them. Given a lack of practical guidance in the literature, we seek to begin the discussion with several case studies and an exploration of four approaches suitable to assessing these emerging services. DESCRIPTION OF PROGRAM This article examines five case studies that vary by staffing, drivers, and institutional context in order to begin a practice-oriented conversation about how to evaluate and assess research data services in academic libraries. The case studies highlight some commonly discussed challenges, including insufficient training and resources, competing demands for evaluation efforts, and the tension between evidence that can be easily gathered and that which addresses our most important questions. We explore reflective practice, formative evaluation, developmental evaluation, and evidence-based library and information practice for ideas to advance practice. NEXT STEPS Data specialists engaged in providing research data services need strategies and tools with which to make decisions about their services. These range from identifying stakeholder needs to refining existing services to determining when to extend and discontinue declining services. While the landscape of research data services is broad and diverse, there are common needs that we can address as a community. To that end, we have created a community-owned space to facilitate the exchange of knowledge and existing resources.

Models for Engaging Liaisons in Research Data Services

“…A similar model was developed at the University of Michigan, with all library liaison roles rewritten to include research data services, along with a core team of librarians with deeper expertise focused on curation, data management, and sharing (Coates et al, 2018). In this model, a working group was charged with providing professional development to all library staff ("in-reach to librarians") to establish a base understanding of research data services, with specific handoff strategies for more specific or advanced questions (Martin & Oehrli, 2015;Coates et al, 2018). Liaisons were assessed to determine if they "felt they were able to provide the services themselves ("do"), provide the service working alongside someone else ("collaborate"), or want to refer a service request to someone else ("refer")" (Coates et al, 2018, 13).…”

Piloting a long-term evaluation of library data workshops

“…The past decade has witnessed growth in the provision of data services by academic health sciences libraries, with new service models including training in research data management, data analysis, and coding for data science in languages like Python and R [1][2][3][4][5][6][7][8][9][10]. In our academic health sciences library, we began offering data services in the form of research data management workshops in 2012.…”

“…After each workshop, we conducted attitudinal surveys of participant experience. We also instituted regular biannual retreats for library faculty and staff to engage in reflective practice [1], but we had not initially done longer-term follow-up with workshop participants.…”

“…Because data services are a relatively new area of medical library work, we found relatively little literature focused on evaluation of data education programming. Coates et al [1] describe the use of reflective practice on the part of librarian-instructors in evaluating workshops, and Deardorff [13] used a mixedmethods study to evaluate the impact of Carpentries workshops on participants. There are also case studies of data education workshops that included descriptions of attitudinal evaluations [1,4,5,8,14] participants demonstrate their learning by making shareable graphics in Google's suite of office tools [9].…”

Research Data Practices of Aerospace Engineering Faculty: A Qualitative Study

“…The results help librarians build a foundation for future DMP services [2]. Coates and others examined five case studies that highlight common challenges for librarians to evaluate existing research data services in academic libraries, and suggested that evidence-based approach provides valuable information for assessing the still-emerging services [3]. Goben and Griffin's study confirmed, "Researchers are most worried about storage, sharing, and issues that revolve around longer term access to data…" Therefore, there are needs for "thoughtfully planned academic RDS [Research Data Service] that are simultaneously broad in scope and strategically focused on addressing specific local needs" [4].…”