Lifting the Veil: Analyzing Collaborative Virtual Reference Transcripts to Demonstrate Value and Make Recommendations for Practice

Brown, Robin

doi:10.5860/rusq.57.1.6441

Cited by 6 publications

(5 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…10 The most common quantitative methods are simple descriptive count-or frequencybased analyses that are accompanied by qualitative coding-based content analyses. 11 In some recent research, advanced quantitative research methods, such as cluster analysis and topic modeling techniques, have been used, but they have not been fully explored yet with a wide range of techniques. 12…”

Section: Chat Transcript Analysis Methods In Library Settingsmentioning

confidence: 99%

Topic Modeling as a Tool for Analyzing Library Chat Transcripts

Koh

Fienup

2021

ITAL

View full text Add to dashboard Cite

Library chat services are an increasingly important communication channel to connect patrons to library resources and services. Analysis of chat transcripts could provide librarians with insights into improving services. Unfortunately, chat transcripts consist of unstructured text data, making it impractical for librarians to go beyond simple quantitative analysis (e.g., chat duration, message count, word frequencies) with existing tools. As a stepping-stone toward a more sophisticated chat transcript analysis tool, this study investigated the application of different types of topic modeling techniques to analyze one academic library’s chat reference data collected from April 10, 2015, to May 31, 2019, with the goal of extracting the most accurate and easily interpretable topics. In this study, topic accuracy and interpretability—the quality of topic outcomes—were quantitatively measured with topic coherence metrics. Additionally, qualitative accuracy and interpretability were measured by the librarian author of this paper depending on the subjective judgment on whether topics are aligned with frequently asked questions or easily inferable themes in academic library contexts. This study found that from a human’s qualitative evaluation, Probabilistic Latent Semantic Analysis (pLSA) produced more accurate and interpretable topics, which is not necessarily aligned with the findings of the quantitative evaluation with all three types of topic coherence metrics. Interestingly, the commonly used technique Latent Dirichlet Allocation (LDA) did not necessarily perform better than pLSA. Also, semi-supervised techniques with human-curated anchor words of Correlation Explanation (CorEx) or guided LDA (GuidedLDA) did not necessarily perform better than an unsupervised technique of Dirichlet Multinomial Mixture (DMM). Last, the study found that using the entire transcript, including both sides of the interaction between the library patron and the librarian, performed better than using only the initial question asked by the library patron across different techniques in increasing the quality of topic outcomes.

show abstract

Section: Chat Transcript Analysis Methods In Library Settingsmentioning

confidence: 99%

Topic Modeling as a Tool for Analyzing Library Chat Transcripts

Koh

Fienup

2021

ITAL

View full text Add to dashboard Cite

show abstract

“…Some studies survey academic librarians about their chat reference services and how they train staff to answer chats (Devine, Paladino, & Davis, 2011), while others explore the usefulness of having full-time librarians provide service through chat reference systems (Maloney & Kemp, 2015). Many chat analysis projects take a large-scale qualitative approach by analyzing datasets of academic chat transcripts to show overall improvements over time (Baumgart et al, 2016;Brown, 2017;Dempsey, 2019). Mungin (2017) at James Madison University analyzed chat transcripts in Dedoose over a five-year span in order to improve chat reference; and as recently as 2019, at Utah State University (which has a high population of students studying online), a group of librarians and learning technologists looked at chat trends over a year by analyzing 1600 chat transcripts through coding.…”

Section: Literature Reviewmentioning

confidence: 99%

Beyond Reference Data: A Qualitative Analysis of Nursing Library Chats to Improve Research Health Science Services

Harlow¹

2021

EBLIP

View full text Add to dashboard Cite

Objective - The objective of this study was to analyze trends in academic library reference chat transcripts with nursing themes, in order to improve all library services and resources based on the findings. Methods - In Fall 2018, health science liaison librarians performed a qualitative study by analyzing 60 nursing chat transcripts from LibraryH3lp. These chats were tagged, anonymized, coded, and then analyzed in Atlas TI to identify patterns and trends. Results - Chat analysis showed that librarians staffing chat are meeting the research needs of nursing patrons by helping them find full-text articles and suggesting the appropriate library databases. In order to further improve these virtual services, workshops were offered to Library and Information Science (LIS) interns and staff who answer reference chats. Nursing online tutorials and research guides were also improved based on the results. Conclusion - This study will help academic libraries improve and expand services into the virtual realm, to support library employees and patrons during the COVID-19 pandemic and beyond. Virtual reference chat is not going away; in the current academic environment it is needed more than ever. Using these library chats as the basis for additional chat staff training can reduce staff anxiety and prepare them to better serve patrons.

show abstract

“…Articles reporting the results of research on digital reference including chat have been published since the 1990s (Matteson et al , 2011). Topics covered in the last few years of literature have ranged from teaching or instruction in chat (Dempsey, 2016, 2017; Hervieux and Tummon, 2018; Jacoby et al , 2016; Schiller, 2016), the types of questions asked in a chat service (Bourgeois and Bealer, 2020; Brown, 2017; Chen and Wang, 2019; Mavodza, 2019; McKewan and Richmond, 2017; Ozeran and Martin, 2019; Stieve and Wallace, 2018), staffing hours or needs, patron satisfaction with the service (Brown, 2017; Mungin, 2017), the role of cobrowsing (Wan et al , 2009) and communication issues (Westbrook, 2007). Probably the largest and most prolific area has involved service quality.…”

Section: Literature Reviewmentioning

confidence: 99%

Constructing citations: reviewing chat transcripts to improve citation assistance as a service

Wilkinson

Denneler

Nay

et al. 2021

RSR

View full text Add to dashboard Cite

PurposeUsing chat transcripts from Indiana University Libraries, the authors examined a subset of transcripts involving citations. From this analysis, they propose improvements for citation assistance as a holistic service.Design/methodology/approachTwo years of chat transcripts were examined and questions containing citation-related keywords were segregated for further examination. The authors used a test data set to create a coding scheme for the questions and responses. This scheme was then applied to all the citation-related transcripts.Findings390 of 11,553 transcripts included interactions about citations. In 42% of the transcripts, no specific citation style was mentioned. American Psychological Association and Modern Language Association were the most frequently mentioned citation styles by chat users. Business reports (company data and market research), periodicals (journal, newspaper or magazine articles), websites and government documents were the most often asked about formats, but there was a wide variety of other unusual formats. Questions about EndNote were more common than other types of citation management software. Chat staff utilized a variety of responses including guiding the student by example, directing to an online resource for more information (85% of the responses) or referring to a citation management expert. An unexpected amount of hedging words in the responses indicates the presence of anxiety on the part of chat staff in responding to these types of questions.Originality/valueThis paper goes beyond most existing studies of chat transcripts by using chat transcripts as data to guide service improvements for a commonly asked but not typically discussed set of questions.

show abstract

Lifting the Veil: Analyzing Collaborative Virtual Reference Transcripts to Demonstrate Value and Make Recommendations for Practice

Cited by 6 publications

References 5 publications

Topic Modeling as a Tool for Analyzing Library Chat Transcripts

Topic Modeling as a Tool for Analyzing Library Chat Transcripts

Beyond Reference Data: A Qualitative Analysis of Nursing Library Chats to Improve Research Health Science Services

Constructing citations: reviewing chat transcripts to improve citation assistance as a service

Contact Info

Product

Resources

About