Predicting Skill Shortages in Labor Markets: A Machine Learning Approach

Abstract

This research develops a Machine Learning approach able to predict labor shortages for occupations. We compile a unique dataset that incorporates both Labor Demand and Labor Supply occupational data in Australia from 2012 to 2018. This includes data from 1.3 million job advertisements (ads) and 20 official labor force measures. We use these data as explanatory variables and leverage the XGBoost classifier to predict yearly labor shortage classifications for 132 standardized occupations. The models we construct achieve macro-F1 average performance scores of up to 86 per cent. However, the more significant findings concern the class of features which are most predictive of labor shortage changes. Our results show that job ads data were the most predictive features for predicting year-to-year labor shortage changes for occupations. These findings are significant because they highlight the predictive value of job ads data when they are used as proxies for Labor Demand, and incorporated into labor market prediction models. This research provides a robust framework for predicting labor shortages, and their changes, and has the potential to assist policy-makers and businesses responsible for preparing labor markets for the future of work.Manuscript submitted for review to the 21st ACM Conference on Economics & Computation (EC'20).