Sentiment Analysis of Unstructured Data Using Spark for Predicting Stock Market Price Movement

被引：2

作者：

Darji, Miss Dhara N. ^{[1
]}

Parikh, Satyen M. ^{[2
]}

Patel, Hiral R. ^{[1
]}

机构：

[1] Ganpat Univ, DCS, Mehsana, India

[2] FCA Ganpat Univ, Mehsana, India

来源：

INVENTIVE COMPUTATION AND INFORMATION TECHNOLOGIES, ICICIT 2021 | 2022年 / 336卷

关键词：

Spark NLP pipeline; Sentiment analysis; Data preprocessing; Stock price movement; TFIDF; Textblob; Vader; Logistic regression; Naive Bayes; Random forest;

D O I：

10.1007/978-981-16-6723-7_39

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

In this digital era, social media generates a large quantity of online financial data, which includes a substantial amount of investor sentiment. On the other hand, only technical and fundamental indicators are no longer adequate to forecast the stock price movement. The investors' sentiments on social media likes, tweets on twitter, comments and post on Facebook as well as other online financial information like online news, google trend, and forum discussion are also affecting the stock price movement. In particular, researchers have gained a lot of interest for analyzing the financial tweets on Twitter and online financial news to study public sentiments. This would be extremely helpful to develop an efficient solution for automating the sentiment analysis of such vast quantities of online financial texts. Henceforth, the proposed sentiment analysis model aims to predict the stock price movement based on the unstructured data like financial tweets on Twitter and news data used, and this research work also introduces Spark NLP-based text preprocessing pipeline to remove noise data and extract the features using the TFDIF by organizing the text in structured format. For sentiment analysis, two library Textblob and Vader are used. Further, the performance comparison has been carried out. The main aim of the proposed sentiment analysis model is to understand the perspective of the writer from a piece of text whether it is positive, negative, or neutral. In an extensivemanner, news and tweets about a security will certainly inspire individuals to invest in that company's stocks, and as a result, the company's stock price will increase.

引用

页码：521 / 530

页数：10

共 12 条

[1]

Das Sushree, 2018, Procedia Computer Science, V132, P956, DOI 10.1016/j.procs.2018.05.111

[2] Sentiment analysis on stock social media for stock price movement prediction [J].

Derakhshan, Ali ;

Beigy, Hamid .

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2019, 85 :569-578

[3] Text Mining Approach to Analyse Stock Market Movement [J].

Elagamy, Mazen Nabil ;

Stanier, Clare ;

Sharp, Bernadette .

INTERNATIONAL CONFERENCE ON ADVANCED MACHINE LEARNING TECHNOLOGIES AND APPLICATIONS (AMLTA2018), 2018, 723 :661-670

[4] The Role of Text Pre-processing in Sentiment Analysis [J].

Haddi, Emma ;

Liu, Xiaohui ;

Shi, Yong .

FIRST INTERNATIONAL CONFERENCE ON INFORMATION TECHNOLOGY AND QUANTITATIVE MANAGEMENT, 2013, 17 :26-32

[5]

Ho KY, 2016, STUD COMPUT INTELL, V628, P395, DOI 10.1007/978-3-319-28495-8_18

[6]

Lee C, 2017, INT CONF AWARE SCI, P312, DOI 10.1109/ICAwST.2017.8256469

[7]

Pagolu V. S., 2016, 2016 INT C SIGN PROC, P1345, DOI [DOI 10.1109/SCOPES.2016.7955659, 10.1109/SCOPES.2016.7955659]

[8] Indian Stock Market Prediction Using Machine Learning and Sentiment Analysis [J].

Pathak, Ashish ;

Shetty, Nisha P. .

COMPUTATIONAL INTELLIGENCE IN DATA MINING, 2019, 711 :595-603

[9]

Pradhan S, 2019, 2019 URSI ASIA-PACIFIC RADIO SCIENCE CONFERENCE (AP-RASC), DOI [10.23919/ursiap-rasc.2019.8738230, 10.1109/ICACCP.2019.8882898, 10.1109/kse.2019.8919368]

[10]

Romanowski A, 2017, STUD COMPUT INTELL, V658, P105, DOI 10.1007/978-3-319-47208-9_7

← 1 2 →