Developing a streaming data processing workflow for querying space–time activities from geotagged tweets

Arteaga, M. Dolores; Cha, Sangwhan; Bourgeois, Yves

doi:10.1016/j.compenvurbsys.2015.12.001

Cited by 11 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are studies showing the power of social media in particular Twitter data in monitoring and transfer data, such as real-time flu and cancer surveillance system by mining Twitter [28], finding patterns related to the health events [29], earthquake reporting system using Twitter as a social sensor for detecting an event in real-time. In [30], a workflow for data ingestion and data management of Twitter streaming data is developed where they retrieved space-time activities from geotagged tweets and stored them in a single cluster of MongoDB. Finding trending topics is discussed in [31].…”

Section: Usage Of Hadoop Technology Authors Inmentioning

confidence: 99%

A scalable and real-time system for disease prediction using big data processing

Ed-daoudy

Maalmi

Ouaazizi

2023

Multimed Tools Appl

View full text Add to dashboard Cite

The growing chronic diseases patients and the centralization of medical resources cause significant economic impact resulting in hospital visits, hospital readmission, and other healthcare costs. This paper proposes a scalable and real-time system for disease prediction from medical data streams. This is carried out by integrating Twitter, Apache Kafka, Apache Spark and Apache Cassandra. Thus, Twitter users tweet attributes related to health, Kafka streaming receives all desired tweets attributes and ingest them to Spark streaming. Here, a machine learning algorithm is applied to predict health status and send back a response message through Kafka. The heart disease dataset, obtained from the UCI repository, was used for experiments. In order to enhance prediction accuracy, Relief algorithm is used for features selection. We compared sex types of relevant machine learning algorithms implemented by Spark MLlib such as Random Forest (RF), Naive Bayes, Support Vector Machine, Multilayer Perceptron, Decision Tree and Logistic Regression with the full features as well as selected features. The highest classification accuracy of 92.05% was reported using RF with selected features. The scalability of RF using Spark MLlib and WEKA framework for both training and application stages was measured. The results show significantly better performances of Spark in terms of scalability and computing times.

show abstract

Section: Usage Of Hadoop Technology Authors Inmentioning

confidence: 99%

A scalable and real-time system for disease prediction using big data processing

Ed-daoudy

Maalmi

Ouaazizi

2023

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…A private tweet involves of one hundred fifty fields on which search can be achieved. Apache-Spark-helps to convert unstructured data into structured data [9].…”

Section: Apache-sparkmentioning

confidence: 99%

“…This research article is completely centered on twitter data analysis. A number of the applications of the actual-time information analytics are surveillance, surroundings, health care, enterprise intelligence, Advertising, visualization, cyber protection, and social media [4][5][6][7][8][9]. This paper provides actual time Statistics framework for studying twitter information.…”

Section: Introductionmentioning

confidence: 99%

Spatial and Sentimental Analysis for Trending Information Technology

Pradhan

Kasetwar

Badjate

2022

SMSJ

View full text Add to dashboard Cite

The users are using Twitter to articulate their opinion. Testimony states that twitter has more than 32.1 crore monthly active user with 10 crore users posting 35 crore of tweets on daily basis. Hence micro-blogging web-sites are a rich originator of data for spatial mining. We performed analysis on trending IT technology. The studies will try to beautify an analytical framework to remove and visualize established and free twitter facts. The projected background contains actual time facts consumption, incessant processing, and facts prophecy factors with Apache-Flume and Spark.

show abstract

“…In this situation, Storm has the role of data filtering and regular processing in real-time. However, using event-processing decrease its latency to sub-second order (almost no latency) but it is not able to use online machine learning [14,15,20].…”

Section: Chapter 2: Related Workmentioning

confidence: 99%

“…There are other studies in stream computing in healthcare applications [14], which focuses upon different sources of healthcare data varying from biomedical images and EHRs to social media data. Wachowicz et al [15] developed a workflow for data ingestion and data management of Twitter streaming data, where they retrieved space-time activities from geotagged tweets and stored them in a single cluster of MongoDB. Other studies [16] search for trends in Twitter posts using complex event-processing (CEP) that may determine important events from a large influx of updates and events and may act upon them in real-time.…”

Section: Chapter 2: Related Workmentioning

confidence: 99%

Developing a Real-Time Data Analytics Framework for Twitter Streaming Data

Yadranjiaghdam

Yasrobi

Tabrizi

2017

2017 IEEE International Congress on Big Data (BigData Congress)

View full text Add to dashboard Cite

Twitter is an online social networking service with more than 300 million users, generating a huge amount of information every day. Twitter's most important characteristic is its ability for users to tweet about events, situations, feelings, opinions, or even something totally new, in real time. Currently there are different workflows offering realtime data analysis for Twitter, presenting general processing over streaming data. This study will attempt to develop an analytical framework with the ability of in-memory processing to extract and analyze structured and unstructured Twitter data. The proposed framework includes data ingestion and stream processing and data visualization components with the Apache Kafka messaging system that is used to perform data ingestion task. Furthermore, Spark makes it possible to perform sophisticated data processing and machine learning algorithms in real time. We have conducted a case study on tweets about the earthquake in Japan and the reactions of people around the world with analysis on the time and origin of the tweets.

show abstract

Developing a streaming data processing workflow for querying space–time activities from geotagged tweets

Cited by 11 publications

References 16 publications

A scalable and real-time system for disease prediction using big data processing

A scalable and real-time system for disease prediction using big data processing

Spatial and Sentimental Analysis for Trending Information Technology

Developing a Real-Time Data Analytics Framework for Twitter Streaming Data

Contact Info

Product

Resources

About