Big data, news diversity and financial market crash

A vast quantity of high-dimensional, unstructured textual news data is produced every day, more than two decades after the launch of the global Internet. These big data have a significant influence on the way that decisions are made in business and finance, due to the cost, scalability, and transparency benefits that they bring. However, limited studies have fully exploited big data to analyze changes in news diversity or to predict financial market movements, specifically stock market crashes. Based on modern methods of textual analysis, this paper investigates the relationship between news diversity and financial market crashes by applying the change-point detection approach. The empirical analysis shows that (1) big data is a relatively new and useful tool for assessing financial market movements, (2) there is a relationship between news diversity and financial market movements. News diversity tends to decline when the market falls and volatility soars, and increases when the market is on an upward trend and in recovery, and (3) the multiple structural breaks detected improve the ability to forecast stock price movements. Therefore, changes to news diversity, embedded in big data, can be a useful indicator of financial market crashes and recoveries.