Machine learning classification of entrepreneurs in British historical census data

Abstract

Thanks to recent data availability, digitized transcriptions of Victorian censuses provide unprecedented historical big data on individuals in the past, but also with new methodological challenges like the classification of otherwise underreported entrepreneurs among a population sample of millions of individuals. This paper presents a methodological solution to accomplish the task of classifying entrepreneurs. We apply machine learning, including deep learning, to outperform a standard logistic regression algorithm. Our methodological developments traverse traditional disciplinary lines using state-of-the-art artificial intelligence methods. The main conclusion of the paper is that significant gains in performance can be achieved with historical archive data through machine learning to test economic theories on historical entrepreneurship. This suggests applicability to other disciplines in information sciences.

A machine learning approach to rural entrepreneurship

“…This is a main feature of the study that contributes to its novelty, as applications of machine learning techniques on the topic of entrepreneurship are very rare. 1 Two recent comprehensive studies that examine the topic are by Montebruno et al (2020) and Sabahi and Parast (2020). Montebruno et al (2020) classify individuals whose entrepreneurial status were unregistered in the British censuses through the period 1851-1881 into the categories of ''entrepreneur '' and ''non-entrepreneur'' (or ''worker'') based on training on newer census data using a wide range of ML algorithms.…”

“…1 Two recent comprehensive studies that examine the topic are by Montebruno et al (2020) and Sabahi and Parast (2020). Montebruno et al (2020) classify individuals whose entrepreneurial status were unregistered in the British censuses through the period 1851-1881 into the categories of ''entrepreneur '' and ''non-entrepreneur'' (or ''worker'') based on training on newer census data using a wide range of ML algorithms. The authors demonstrate how ML algorithms outperform traditional classification methods in the context of a research question similar to that of the present study, albeit with a much smaller number of features.…”

“…The authors demonstrate how ML algorithms outperform traditional classification methods in the context of a research question similar to that of the present study, albeit with a much smaller number of features. This being said, the historical data used by Montebruno et al (2020) contains information related to occupation/industry which clearly can improve predictions when used predicting entrepreneurial success. While this information is also present in LiTS III (for the respondent and the parents), the large number of missing observations for our restricted sample renders it unusable.…”

“…2 Sabahi and Parast (2020), on the other hand, take a different approach and establish a clear connection between project performance and individual entrepreneurial orientation, by considering entrepreneurship as an explanatory feature is a series of ML algorithms among other predictors. Clearly, studies by Montebruno et al (2020) and Sabahi and Parast (2020) are important illustrations on the suitability of ML methods to understand the underlying mechanisms of entrepreneurial activity, particularly given the high chance that the outcome is subject to ''ambiguous functional forms'' (Gu et al, 2018;Sabahi & Parast, 2020). Nevertheless, despite the existence of such important contributions, the scarcity of the application of ML methods is also apparent in the area of economic research as a whole (Athey & Imbens, 2019).…”

Advances in Machine Learning Algorithms for Hate Speech Detection in Social Media: A Review

“…It is evident that each model has its share of pitfalls; therefore, no model is perfect. Though the ensemble methods try to add up the advantages of other models together to give a better performance than any single model can offer [63]. Statistically speaking, combining two or more ML algorithms can generally reduce their variance and significantly improve their learning capabilities [64].…”