Iktishaf: a Big Data Road-Traffic Event Detection Tool Using Twitter and Spark Machine Learning

Alomari, Ebtesam; Katib, Iyad; Mehmood, Rashid

doi:10.1007/s11036-020-01635-y

Cited by 52 publications

(62 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This component is part of our earlier paper, Iktishaf. For further details, see the pre-processing algorithm in [ 6 ].…”

Section: The System Methodology and Designmentioning

confidence: 99%

“…That leads to minimizing the word confusion and losing or changing the word meaning. For further details, see the stemmer algorithm in [ 6 ].…”

Section: The System Methodology and Designmentioning

confidence: 99%

“…Artificial intelligence has taken us by storm [ 24 ], and has led to the emergence of concepts such as artificially intelligent cities [ 25 ]. A key to providing smartness for emerging urban and rural environments is to continuously sense and analyze these environments and make timely and effective decisions [ 6 , 10 , 24 , 26 ]. Social media analysis using machine learning has become a key method to provide the pulse for sensing and engaging with the environments [ 6 ] and is expected to provide smarter solutions for our fight against COVID-19 and future pandemics as well as during peace times.…”

Section: Literature Reviewmentioning

confidence: 99%

“…The software developed for this work is part of the tool Iktishaf [ 6 , 7 , 8 , 9 ] that we have been developing for the last few years. Earlier work on this tool has focused mainly on mobility-related event detection using supervised learning.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

COVID-19: Detecting Government Pandemic Measures and Public Concerns from Twitter Arabic Data Using Distributed Machine Learning

Alomari

Katib

Albeshri

et al. 2021

IJERPH

Self Cite

View full text Add to dashboard Cite

Today’s societies are connected to a level that has never been seen before. The COVID-19 pandemic has exposed the vulnerabilities of such an unprecedently connected world. As of 19 November 2020, over 56 million people have been infected with nearly 1.35 million deaths, and the numbers are growing. The state-of-the-art social media analytics for COVID-19-related studies to understand the various phenomena happening in our environment are limited and require many more studies. This paper proposes a software tool comprising a collection of unsupervised Latent Dirichlet Allocation (LDA) machine learning and other methods for the analysis of Twitter data in Arabic with the aim to detect government pandemic measures and public concerns during the COVID-19 pandemic. The tool is described in detail, including its architecture, five software components, and algorithms. Using the tool, we collect a dataset comprising 14 million tweets from the Kingdom of Saudi Arabia (KSA) for the period 1 February 2020 to 1 June 2020. We detect 15 government pandemic measures and public concerns and six macro-concerns (economic sustainability, social sustainability, etc.), and formulate their information-structural, temporal, and spatio-temporal relationships. For example, we are able to detect the timewise progression of events from the public discussions on COVID-19 cases in mid-March to the first curfew on 22 March, financial loan incentives on 22 March, the increased quarantine discussions during March–April, the discussions on the reduced mobility levels from 24 March onwards, the blood donation shortfall late March onwards, the government’s 9 billion SAR (Saudi Riyal) salary incentives on 3 April, lifting the ban on five daily prayers in mosques on 26 May, and finally the return to normal government measures on 29 May 2020. These findings show the effectiveness of the Twitter media in detecting important events, government measures, public concerns, and other information in both time and space with no earlier knowledge about them.

show abstract

“…This component is part of our earlier paper, Iktishaf. For further details, see the pre-processing algorithm in [ 6 ].…”

Section: The System Methodology and Designmentioning

confidence: 99%

“…That leads to minimizing the word confusion and losing or changing the word meaning. For further details, see the stemmer algorithm in [ 6 ].…”

Section: The System Methodology and Designmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

COVID-19: Detecting Government Pandemic Measures and Public Concerns from Twitter Arabic Data Using Distributed Machine Learning

Alomari

Katib

Albeshri

et al. 2021

IJERPH

Self Cite

View full text Add to dashboard Cite

show abstract

“…These micro data centers also allow creating batches of data to add the network latency in data transfers to a set of events rather than to each event [10]. Micro-batch is an attractive method in SP because it permits to achieve better throughput and application latency [11]. Previous works attempt to employ micro-batch strategies using an absolute number of events [12], [13] or time [14] for creating the batches.…”

Section: Introductionmentioning

confidence: 99%

Boosting Big Data Streaming Applications in Clouds With BurstFlow

et al. 2020

View full text Add to dashboard Cite

An efficient approach for detecting anomalous events in real‐time weather datasets

Arora

Rani

Saxena

2021

Concurrency and Computation

View full text Add to dashboard Cite

Event detection in real‐time is applied in diverse domains such as detection of fraudulent activities in commercial transactions, detection of faulty systems in industries, and so forth. Businesses and organizations benefit from the actionable information obtained through various techniques available for anomalous event detection. Real‐time event detection is nowadays handled through streaming data frameworks. Traditional approaches effectively handle event detection in real‐time but with more false positives, thus, resulting in false alarms. In this article, an efficient approach comprising two components, an offline model and an online event detection pipeline, is proposed to achieve minimum mean absolute error (MAE). An offline module is developed to investigate a variety of deep learning models that prove suitable for event detection in real‐time. The experiments conducted with PubNub sensors datasets demonstrate that the long short‐term memory unit of recurrent neural networks is the best suitable model for anomalous event detection. The online pipeline module is built using streaming data frameworks to predict the abnormal peaks. It is revealed through the experimental results that the proposed approach efficiently detects anomalous events in real‐time and also eliminates false positives.

show abstract

Iktishaf: a Big Data Road-Traffic Event Detection Tool Using Twitter and Spark Machine Learning

Cited by 52 publications

References 55 publications

COVID-19: Detecting Government Pandemic Measures and Public Concerns from Twitter Arabic Data Using Distributed Machine Learning

COVID-19: Detecting Government Pandemic Measures and Public Concerns from Twitter Arabic Data Using Distributed Machine Learning

Boosting Big Data Streaming Applications in Clouds With BurstFlow

An efficient approach for detecting anomalous events in real‐time weather datasets

Contact Info

Product

Resources

About