Unstructured Data Analysis on Big Data Using Map Reduce

Abstract

In the real time scenario, the volume of data used linearly increases with time. Social networking sites like Facebook, Twitter discovered the growth of data which will be uncontrollable in the future. In order to manage the huge volume of data, the proposed method will process the data in parallel as small chunks in distributed clusters and aggregate all the data across clusters to obtain the final processed data. In Hadoop framework, MapReduce is used to perform the task of filtering, aggregation and to maintain the efficient storage structure. The data are preferably refined using collaborative filtering, under the prediction mechanism of particular data needed by the user. The proposed method is enhanced by using the techniques such as sentiment analysis through natural language processing for parsing the data into tokens and emoticon based clustering. The process of data clustering is based on user emotions to get the data needed by a specific user. The results show that the proposed approach significantly increases the performance of complexity analysis.

An Approach To Twitter Sentiment Analysis Over Hadoop

“…It is observed that by almost 3% the MaxEnt trained from a combination of unigrams and bigrams outperforms other models trained from a combination of POS tags and unigrams. Subramaniyaswamy et al [14] in this paper deals with Hash Algorithm, MRAP (Map-Reduce Access Patterns), Hadoop Map reduce and Collaborative Filtering. In this research work, firstly the unstructured data is structured and then processed by using Map Reduce technique and after that the automatic prediction of user"s taste is done through collaborative filtering.…”

Sentiment Analysis of Online Product Reviews Using DLMNN and Future Prediction of Online Product Using IANFIS

“…Along with it, the reviews for the product can as well be seen in Social networking sites [2]. In present years, social networks have turned very popular; so there is a chance that because of those sites, the expansion of data can be uncontrollable in the future [3,4]. Since, everyone is posting comments every day, which led to the immense continuous augmentation in the online data and online information.…”

Effective processing of unstructured data using python in Hadoop map reduce

“…An examination work in 2011 exhibited that 92% of the world's whole information had been made inside the most recent six years [1]. In starting of 2004, the wide amount of data was maintained on single server.…”

“…Unstructured information does not have an organized structure. The interpersonal interaction media widely gives the services to real world with advanced features like Time Saving & an Attractive way [1]. Since the present scenario data are unstructured, data should be process effectively, and managed the data to satisfy our future needs.…”