An Empirical Investigation of Bias in the Multimodal Analysis of Financial Earnings Calls

Abstract

Volatility prediction is complex due to the stock market's stochastic nature. Existing research focuses on the textual elements of financial disclosures like earnings calls transcripts to forecast stock volatility and risk, but ignores the rich acoustic features in the company executives' speech. Recently, new multimodal approaches that leverage the verbal and vocal cues of speakers in financial disclosures significantly outperform previous stateof-the-art approaches demonstrating the benefits of multimodality and speech. However, the financial realm is still plagued with a severe underrepresentation of various communities spanning diverse demographics, gender, and native speech. While multimodal models are better risk forecasters, it is imperative to also investigate the potential bias that these models may learn from the speech signals of company executives. In this work, we present the first study to discover the gender bias in multimodal volatility prediction due to gendersensitive audio features and fewer female executives in earnings calls of one of the world's biggest stock indexes, the S&P 500 index. We quantitatively analyze bias as error disparity and investigate the sources of this bias. Our results suggest that multimodal neural financial models accentuate gender-based stereotypes. 1

Speech emotion recognition and text sentiment analysis for financial distress prediction

“…In a similar manner, a multi-modal prediction model also worked well in predicting stock price volatility Sawhney et al [50], as well as the neural attentive alignment model capturing interdependencies between verbal and vocal modalities [49,51]. The most recent work found that there is gender bias in multi-modal financial forecasts [52] and that managers modify the way they talk knowing that machines are listening [10].…”

Numerical Claim Detection in Finance: A Weak-Supervision Approach

“…Moreover, multimodal machine learning was used by [2] and [43] for credit rating prediction and measurement of persuasiveness respectively. [1] investigated biases in the multimodal analysis of financial earnings calls. Finally, [44] provide critical analysis of how corporate disclosure has been reshaped over last couple of years due to increasing use of NLP in Finance.…”

“…Quarterly analyst reports (in English) on a large number of public firms in the U.S. constitute the raw dataset for our model. These analysts reports were collected from Zacks Equity Research and were available to us from Nexis Uni license 1 . Before the data is passed on to labelling functions 1.…”

“…The surge in machine learning and its applications has opened up a new arena of possibilities in diverse fields ranging from image recognition, natural language processing to finance [1], [2], [3], [4], [5]. However, a major challenge for building or training predictive models is the scarcity of labelled data [6], [7].…”

Multimodal Multi-Speaker Merger & Acquisition Financial Modeling: A New Task, Dataset, and Neural Baselines

“…We acknowledge the presence of gender bias in our study, given the imbalance in the gender ratio of speakers of the calls. We also acknowledge the demographic bias (Sawhney et al, 2021a) in our study as the companies are organizations within the public stock market of United States of America and may not generalize directly to non-native speakers.…”