Twitter Data Analysis Using Hadoop and ‘R’ and Emotional Analysis Using Optimized SVNN

Abstract

Standalone systems cannot handle the giant traffic loads generated by Twitter due to memory constraints. A parallel computational environment provided by Apache Hadoop can distribute and process the data over different destination systems. In this paper, the Hadoop cluster with four nodes integrated with RHadoop, Flume, and Hive is created to analyze the tweets gathered from the Twitter stream. Twitter stream data is collected relevant to an event/topic like IPL-2015, cricket, Royal Challengers Bangalore, Kohli, Modi, from May 24 to 30, 2016 using Flume. Hive is used as a data warehouse to store the streamed tweets. Twitter analytics like maximum number of tweets by users, the average number of followers, and maximum number of friends are obtained using Hive. The network graph is constructed with the user's unique screen name and mentions using 'R'. A timeline graph of individual users is generated using 'R'. Also, the proposed solution analyses the emotions of cricket fans by classifying their Twitter messages into appropriate emotional categories using the optimized support vector neural network (OSVNN) classification model. To attain better classification accuracy, the performance of SVNN is enhanced using a chimp optimization algorithm (ChOA). Extracting the users' emotions toward an event is beneficial for prediction, but when coupled with visualizations, it becomes more powerful. Bar-chart and wordcloud are generated to visualize the emotional analysis results.

Practice and Application of Fusion Machine Learning in Data Analysis

“…For example, some experts point out that machine learning enables machines to learn rules from various data to classify and recognize new samples. [1][2] Some experts also designed and implemented a LIS data analysis system based on web data mining and concurrent access control technology [3][4]. In addition, some experts believe that data warehouse and data mining technology can better organize big data, make it play its best role, and provide a basis for business decision-making [5][6].…”