Text Classification for Organizational Researchers

Abstract

Organizations are increasingly interested in classifying texts or parts thereof into categories, as this enables more effective use of their information. Manual procedures for text classification work well for up to a few hundred documents. However, when the number of documents is larger, manual procedures become laborious, time-consuming, and potentially unreliable. Techniques from text mining facilitate the automatic assignment of text strings to categories, making classification expedient, fast, and reliable, which creates potential for its application in organizational research. The purpose of this article is to familiarize organizational researchers with text mining techniques from machine learning and statistics. We describe the text classification process in several roughly sequential steps, namely training data preparation, preprocessing, transformation, application of classification techniques, and validation, and provide concrete recommendations at each step. To help researchers develop their own text classifiers, the R code associated with each step is presented in a tutorial. The tutorial draws from our own work on job vacancy mining. We end the article by discussing how researchers can validate a text classification model and the associated output.

Quantifying with words: An investigation of the validity of narrative‐derived performance scores

“…Although Kobayahsi et al. () suggest that these features might not improve model performance, within this study, the inclusion of bigrams and trigrams was found to improve model performance in cross‐validated samples ( R 2 was 1.7% higher than a model using just unigrams), thus justifying their use.…”

“…() and recommended by Kobayashi et al. (). Second, the number of vectors was further reduced by only including terms that correlated significantly with job performance (i.e., univariate feature selection) using a conservative value of p < .001.…”

“…Natural language processing is a field of linguistic science that seeks to understand how computers can comprehend and mimic human language (Landers, 2017;Manning & Schütze, 1999); this includes a series of methods to classify, mine, and predict information from text data, otherwise known as text mining or text analytics. Text mining analyses range from fully supervised (i.e., keyed upon some criterion, such as expert ratings) to fully unsupervised approaches (Pang & Lee, 2008) and are used for a range of analytical tasks, including topical categorization, thematic analysis, sentiment analysis, author attribution, and personality measurement, among many others (e.g., Grimmer & Stewart, 2013;Ignatow & Mihalcea, 2017;Kobayashi et al, 2017aKobayashi et al, , 2017bLiu, 2012;Mairesse, Walker, Mehl, & Moore, 2007;Medhat, Hassan, & Korashy, 2014;Schwartz et al, 2013;Stamatatos, 2009). Text mining has increasingly been applied to the social sciences (e.g., Ignatow & Mihalcea, 2017;McAbee, Landis, & Burke, 2017), although within this domain, the practice is still in its infancy.…”

“…The resulting document term matrix and its features were then reduced in two ways. First, any terms found in less than 1% of the documents were removed (i.e., removing sparse terms), similar to the approach taken by Schwartz et al (2013) and recommended by Kobayashi et al (2017a). Second, the number of vectors was further reduced by only including terms that correlated significantly with job performance (i.e., univariate feature selection) using a conser-…”

The application of text mining methods in innovation research: current state, evolution patterns, and development priorities

“…Scholars have used qualitative approaches to text analysis including manual coding, discourse analytical methods, or grounded theory (Duriau et al, 2007). These manual procedures, however, are labor intensive and seem to have reached their natural limits when it comes to analyzing increasingly large amounts of text material (Jamiy et al, 2015; Kobayashi et al, 2018). Consequently, researchers have started to explore the opportunities that the computer‐aided, or automated, analysis of textual data offers (Janasik et al, 2009; Wiedemann, 2013).…”