Distributed real-time sentiment analysis for big data social streams

Rahnama, Amir Hossein Akhavan

doi:10.1109/codit.2014.6996998

Cited by 19 publications

(11 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cheng et al, in their work, developed a framework to analyze consumer opinions about products using Apache Hadoop and the Hadoop stream processor, Storm. In the works of Rahnama, tweets are analyzed to discover their sentiment with the use of the Hadoop‐Storm pair. This study compares tweet sentiment classification performance with the use of scalable classifiers and uniprocessor algorithms.…”

Section: Related Workmentioning

confidence: 99%

“…Considering the conventional SA research works and frameworks in the literature, there exists many SA studies related to different languages. However, infrequent research works (mostly in English) are conducted about real‐time SA in the literature . There is an increasing need to analyze real‐time stream data such as spot and stop fraudulent activity in the finance domain, inventory management, web analytics/content management (sales performance evaluation through searched keywords), and real‐time customer behavior analysis to improve customer experience …”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Kılınç

2019

Softw Pract Exp

View full text Add to dashboard Cite

Summary There are many data sources that produce large volumes of data. The Big Data nature requires new distributed processing approaches to extract the valuable information. Real‐time sentiment analysis is one of the most demanding research areas that requires powerful Big Data analytics tools such as Spark. Prior literature survey work has shown that, though there are many conventional sentiment analysis researches, there are only few works realizing sentiment analysis in real time. One major point that affects the quality of real‐time sentiment analysis is the confidence of the generated data. In more clear terms, it is a valuable research question to determine whether the owner that generates sentiment is genuine or not. Since data generated by fake personalities may decrease accuracy of the outcome, a smart/intelligent service that can identify the source of data is one of the key points in the analysis. In this context, we include a fake account detection service to the proposed framework. Both sentiment analysis and fake account detection systems are trained and tested using Naïve Bayes model from Apache Spark's machine learning library. The developed system consists of four integrated software components, ie, (i) machine learning and streaming service for sentiment prediction, (ii) a Twitter streaming service to retrieve tweets, (iii) a Twitter fake account detection service to assess the owner of the retrieved tweet, and (iv) a real‐time reporting and dashboard component to visualize the results of sentiment analysis. The sentiment classification performances of the system for offline and real‐time modes are 86.77% and 80.93%, respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Kılınç

2019

Softw Pract Exp

View full text Add to dashboard Cite

show abstract

“…For this reason, the 5Vs theme in big data is revisited. Several literatures have started to explore the big data issue for SA, such as for the scalability issue (Bing and Chan, 2014;Conejero et al, 2013;Liu et al, 2013), introduction of big data tools for SA (Ding et al, 2013;Mihanović et al, 2014;Prom-on et al, 2014), distributed approach for SA processing (Bravo-Marquez et al, 2014;Fulse et al, 2014;Hossein and Rahnama, 2014) and improved ML models for SA on big data (Bing and Chan, 2014;Ding et al, 2013;Liu et al, 2013;Mukkamala et al, 2014). Undoubtedly, these papers are dated around the year 2014, which marks the booming of the big data era.…”

Section: Gaps and Opportunities Between Sentiment Analysis Approachesmentioning

confidence: 99%

Overview and Future Opportunities of Sentiment Analysis Approaches for Big Data

Sharef¹,

Zin²,

Nadali³

2016

Journal of Computer Science

View full text Add to dashboard Cite

Abstract:The ability to exploit public sentiment in social media is increasingly considered as an important tool for market understanding, customer segmentation and stock price prediction for strategic marketing planning and manoeuvring. This evolution of technology adoption is energised by the healthy growth in big data framework, which caused applications based on Sentiment Analysis (SA) in big data to become common for businesses. However, scarce works have studied the gaps of SA application in big data. The contribution of this paper is two-fold: (i) this study reviews the state of the art of SA approaches. including sentiment polarity detection, SA features (explicit and implicit), sentiment classification techniques and applications of SA and (ii) this study reviews the suitability of SA approaches for application in the big data frameworks, as well as highlights the gaps and suggests future works that should be explored. SA studies are predicted to be expanded into approaches that utilise scalability, possess high adaptability for source variation, velocity and veracity to maximise value mining for the benefit of the users.

show abstract

“…For the second use case, a stream processing platform where data producers are continuously sending data at variable rates is considered. An example of such applications is continuous mining of social network data such as, analysis of twitter streams for trend analysis and sentiment analysis [75,76]. Usually, social networking platforms expose APIs to tap on their stream data and continuously push the streams to a data analytics platform.…”

Section: Use Case-2: Systems Subjected To the Arrival Of Continuous Datamentioning

confidence: 99%

“…While continuous processing of tuples can correspond to a number operations such as monitoring, analytics, ads targeting, data synchronization [80], sentiment and trend analysis of twitter feeds [75,76], the topologies in Use Case 2 are not tied to any specific computation. Instead processing time for each tuple is characterized by a parameter called service time which is discussed in Section 4.2.5.…”

Section: Storm Topologiesmentioning

confidence: 99%

Priority-Based Scheduling Techniques for a Multitenant Stream Processing Platform

Chakraborty¹

View full text Add to dashboard Cite

Apache Storm is a popular distributed stream processing system which has been widely adopted by the key players in the industry including YAHOO and Twitter. An application running in Storm is called a topology that is characterized by a Directed Acyclic Graph.Isolation Scheduler, the default scheduler for a multitenant Storm platform running multiple topologies assigns resources to topologies based on static resource configuration information and does not provide any means to prioritize topologies based on their business significances. One of the problems with this scheduler is that, performance degradation even complete starvation of topologies is possible on a resource constrained cluster. Two priority based resource scheduling strategies are proposed in this thesis to overcome these problems. A performance analysis based on prototyping and measurements is conducted to demonstrate the effectiveness of the proposed techniques. A comprehensive analysis of the results leading to key insights into system behavior and performance is presented.iii Acknowledgements First and foremost, I would like to thank my supervisor, Dr. Shikharesh Majumdar, for his continuous guidance, support, and encouragement during the process of writing this thesis and over the duration of this program. I sincerely appreciate his invaluable feedbacks and patience during every step of this thesis.

show abstract

Distributed real-time sentiment analysis for big data social streams

Cited by 19 publications

References 9 publications

A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Overview and Future Opportunities of Sentiment Analysis Approaches for Big Data

Priority-Based Scheduling Techniques for a Multitenant Stream Processing Platform

Contact Info

Product

Resources

About