Insight workflow: Systematically combining human and computational methods to explore textual data

Abstract

. Insight workflow: Systematically combining human and computational methods to explore textual data. Journal of the Association for Information Science and Technology, 68(7), 1671 -1686 . DOI: 10.1002 Citing this paper Please note that where the full-text provided on King's Research Portal is the Author Accepted Manuscript or Post-Print version this may differ from the final Published version. If citing, it is advised that you check and use the publisher's definitive version for pagination, volume/issue, and date of publication details. And where the final published version is provided on the Research Portal, if citing you are again advised to check the publisher's website for any subsequent corrections. General rightsCopyright and moral rights for the publications made accessible in the Research Portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognize and abide by the legal requirements associated with these rights.•Users may download and print one copy of any publication from the Research Portal for the purpose of private study or research.•You may not further distribute the material or use it for any profit-making activity or commercial gain •You may freely distribute the URL identifying the publication in the Research Portal Take down policyIf you believe that this document breaches copyright please contact librarypure@kcl.ac.uk providing details, and we will remove access to the work immediately and investigate your claim. AbstractAnalysing large quantities of real world textual data has the potential to provide new insights for researchers. However, such data present challenges for both human and computational methods, requiring a diverse range of specialist skills, often shared across a number of individuals. In this paper, we use the analysis of a real world data set as our case study, and use this exploration as a demonstration of our 'insight workflow', which we present for use and adaptation by other researchers. The data we use are impact case study documents collected as part of the UK Research Excellence Framework (REF), consisting of 6,679 documents and 6.25 million words; the analysis was commissioned by the Higher Education Funding Council for England (published as report HEFCE 2015). In our exploration and analysis we used a variety of techniques, ranging from keyword in context and frequency information to more sophisticated methods (topic modelling), with these automated techniques providing an empirical point of entry for in depth and intensive human analysis. We present the 60 topics to demonstrate the output of our methods, and illustrate how the variety of analysis techniques can be combined to provide insights. We note potential limitations and propose future work. SYSTEMATICALLY EXPLORING TEXT 3

Product Review-Based Customer Sentiment Analysis Using an Ensemble of mRMR and Forest Optimization Algorithm (FOA)

“…The dataset is derived from Amazon that has been developed by John Blitzer and Mark Dredges to analysis the sentiments of customers used for classification purpose. (Gill et al, 2017) This dataset contains a variety of products and customers reviews, including electronic devices, kitchen appliances, books, DVDs and so on. All reviews are in a raw structure with HTML tags of review content, review identification number, date, and title, rating, product, and the location of customer.…”

Research knowledge utilisation for societal impact: Information practices based on abductive topic modelling

“…All 6637 available cases were downloaded from the REF 2014 impact case studies website (available at: https:// impact.ref.ac.uk/casestudies/). Following prior literature on analysing the REF case studies [51], our analysis focused on the fourth section of the case studies (i.e. details of the impact) in that it provides a detailed description of how research knowledge embedded in the research outputs is utilised outside academia for achieving societal impact.…”

“…In addition, eight specific stop words that were very frequently used in the impact cases were also excluded (i.e. 'impact', 'ref', 'date', 'new', 'case', 'study', 'page', 'research') because they did not add any insights to our understanding of this corpus [51]. Finally, we performed a stemming step to transform different forms of the same word (e.g.…”

“…The font size in each word is proportional to the frequency of each word. These raw word frequencies only indicate the ‘popularity’ of a word within the corpus [51]. The co-occurrence between these singular words will be identified using topic modelling in the following steps.…”

Applying Computational Analysis to Textual Data from the Wild

“…The potential for placing automated topic analysis alongside more qualitative approaches has gained traction, particularly for research in sociology [56]. The approach described in this paper is most aligned with Gill et al [15], which combines techniques such as keyword in context, frequency information and topic modelling as an empirical point of entry for human analysis. In particular, we are exploring the suitability of such methods for interpreting geo-referenced textual data.…”