Shared disk big data analytics with Apache Hadoop

Mukherjee, Anirban; Datta, Joydip; Jorapur, Raghavendra; Singhvi, Ravi; Haloi, Saurav; Akram, Wasim

doi:10.1109/hipc.2012.6507520

Cited by 22 publications

(7 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While executing the Map Reduce framework, [5] Hadoop splits the process into map and reducer task that performs operations such as issuing task, checks the completion of the task, replications of data and data transformation from one cluster to another. YouTube is used to promote various company through ads.…”

Section: II Related Workmentioning

confidence: 99%

Mining and YouTube Data Analysis using Hadoop

2020

IJITEE

View full text Add to dashboard Cite

Analysis of structured and consistent data has seen remarkable success in past decades. Whereas, the analysis of unstructured data in the form of multimedia format remains a challenging task. YouTube is one of the most popular and used social media tool. It reveals the community feedback through comments for published videos, number of likes, dislikes, number of subscribers for a particular channel. The main objective of this work is to demonstrate by using Hadoop concepts, how data generated from YouTube can be mined and utilized to make targeted, real time and informed decisions. In our paper, we analyze the data to identify the top categories in which the most number of videos are uploaded. This YouTube data is publicly available and the YouTube data set is described below under the heading Data Set Description. The dataset will be fetched from the Google using the YouTube API (Application Programming Interface) and going to be stored in Hadoop Distributed File System (HDFS). Using MapReduce we are going to analyze the dataset to identify the video categories in which most number of videos are uploaded. The objective of this paper is to demonstrate Apache Hadoop framework concepts and how to make targeted, real-time and informed decisions using data gathered from YouTube.

show abstract

Section: II Related Workmentioning

confidence: 99%

Mining and YouTube Data Analysis using Hadoop

2020

IJITEE

View full text Add to dashboard Cite

show abstract

“…As the storage backend consists of a single Storage Area Network (HPE 3PAR StorServ) with multiple disks, it is also important to test and benchmark the throughput of the HDFS. The benchmarking has been performed in conjunction with several suggestions provided by Mukherjee et al [40]. It is appropriate to use a distributed file system such as HDFS on top of a shared disk infrastructure such as a storage area network.…”

Section: Benchmarking the Hadoop Cluster (I/o)mentioning

confidence: 99%

Multi-dimensional geospatial data mining in a distributed environment using MapReduce

Alkathiri

Abdul²,

Potdar³

2019

J Big Data

View full text Add to dashboard Cite

Data mining and machine learning techniques for processing raster data consider a single spectral band of data at a time. The individual results are combined to obtain the final output. The essence of related multi-spectral information is lost when the bands are considered independently. The proposed platform is based on Apache Hadoop ecosystem and supports performing analysis on large amounts of multispectral raster data using MapReduce. A novel technique of transforming the spectral space to the geometrical space is also proposed. The technique allows to consider multiple bands coherently. The results of clustering 10 6 pixels for multiband imagery with widely used GIS software have been tested and other machine learning methods are planned to be incorporated in the platform. The platform is scalable to support tens of spectral bands. The results from our platform were found to be better and are also available faster due to application of distributed processing.

show abstract

“…Hence, the data clustering will be handled under distributed Hadoop environment which serves choice in crop planning by forecast the demand in the market at the earliest [3]. In this paper, a popular Map-Reduce concept utilized clustered file system extensively with Hadoop Distributed File System (HDFS) [4]. The purpose behind the Map-Reduce paradigm is high scalable which executes massively parallel and distributed over a huge number of computing nodes [5].…”

Section: Hadoop Terminologymentioning

confidence: 99%

Big Data Analytics on Indian Crop Planning to Increase Agricultural Production

Sekhar¹,

Kumar²,

Kumar³

et al. 2017

Advanced Science and Technology Letters

View full text Add to dashboard Cite

Abstract. To find a hidden pattern in crop sales by applying data clustering on it. By using time or crop type as the clustering factor, to predict the schedule of crop sowing/planting or decide which crop should be sown/planted in the season. If the pattern of such crops are already known, one can set the plan, when to harvest the product, how much the price and how many. To show results, data clustering will be used to analyze the data pattern, crop pattern and crop sales pattern. Besides, it predicts the price of crops in further years or a season which helps farmers to adopt the crop cultivation plan. This paper emphasizes the use of Big Data (BD) which helps the farmers will upgrade the worth of their products via less pesticides to be focused.

show abstract

Shared disk big data analytics with Apache Hadoop

Cited by 22 publications

References 2 publications

Mining and YouTube Data Analysis using Hadoop

Mining and YouTube Data Analysis using Hadoop

Multi-dimensional geospatial data mining in a distributed environment using MapReduce

Big Data Analytics on Indian Crop Planning to Increase Agricultural Production

Contact Info

Product

Resources

About