Textual analysis of stock market prediction using breaking financial news

Abstract

Our research examines a predictive machine learning approach for financial news articles analysis using several different textual representations: bag of words, noun phrases, and named entities. Through this approach, we investigated 9,211 financial news articles and 10,259,042 stock quotes covering the S&P 500 stocks during a five week period. We applied our analysis to estimate a discrete stock price twenty minutes after a news article was released. Using a support vector machine (SVM) derivative specially tailored for discrete numeric prediction and models containing different stock-specific variables, we show that the model containing both article terms and stock price at the time of article release had the best performance in closeness to the actual future stock price (MSE 0.04261), the same direction of price movement as the future price (57.1% directional accuracy) and the highest return using a simulated trading engine (2.06% return). We further investigated the different textual representations and found that a Proper Noun scheme performs better than the de facto standard of Bag of Words in all three metrics.

A quantitative stock prediction system based on financial news

“…This representation was found to be useful by removing the ambiguity associated with particular proper nouns which could either be represented by more than one named entity or fall outside one of the seven defined Named Entity categories. In a comparison study using these four representational techniques, it was found that Proper Noun representation was more effective in symbolizing financial news articles (Schumaker & Chen, 2006).…”

“…This component has derived from prior empirical testing and includes article term representations and the stock price at the time the news article was released. This combination of parameters was previously tested and judged to provide superior performance to all combinations tested (Schumaker & Chen, 2006). The fourth major component is the Trading Experts which gathers the daily buy/sell recommendations from a variety of trading experts.…”

“…To test the effects of GICS partitioning, we trained our system on keywords for all stocks, each GICS Sector, Industry Group, Industry and SubIndustry, as well as trained on articles for each specific company. Output from these models are then evaluated on a three metric platform consistent with prior research (Schumaker & Chen, 2006).…”

How The Market Can Detect Its Own Mispricing - A Sentiment Index To Detect Irrational Exuberance

“…Following the idea of Hagenau's paper's title 'Reading All The News at the Same Time', we analyze news to extract sentiment values. The focus in doing so is not based on how a breaking news announcement influences the short-term stock return [17], [18], or to predict the stock markets volatility [19]. Nether do we test noise trade methods, based on sentiment data [20].…”

Expert Stock Picker: The Wisdom of (Experts in) Crowds

“…UGC has been used in aggregate to predict recommendation system ratings [15], music sales [6], and blockbuster performance [8]. User-generated text has also been used to predict stock market performance [2,18,25].…”