Interpretable deep learning based text regression for financial prediction

Abstract

Text regression is an important task in natural language processing (NLP), which aims to predict continuous numerical values associated with text. Previous work focused on linear text regression requiring manual feature selection for financial prediction. Recently, non-linear text regression through neural network models has become a trend. However, most models rely only on convolutional neural networks (CNN) and suffer from insufficient interpretability. In this paper, we propose a deep neural network model named EM-CBA for text regression and further interpret the model. The proposed model is powered by word EMbedding, CNN, Bidirectional long short-term memory (Bi-LSTM) and Attention mechanism. The proposed EM-CBA takes financial report texts as input and predicts a financial metric named return on assets (ROA). We conduct comprehensive experiments on a dataset about the reports of enterprises. Experimental results show that the proposed model provides more accurate predictions of enterprises' metrics than previous convolutional neural network models and other classical models. The validity of each module of the model is also verified. Finally, we demonstrate a way of performing analysis in words change and results errors to intuitively interpret the effect of different text inputs on the model. The analysis demonstrates that the model is able to use information about sentiment words to analyse their associated contexts to revise the predictions.