Extending Storm API for Real Time Computing on Big Data

Munshi, Shiladitya; Saha, Sajal; Goswami, Radha Tamal

doi:10.1109/indicon47234.2019.9028971

Cited by 1 publication

(2 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Though standard of Apache HadoopTM has evolved as a batch processing environment, still researchers have reported their studies in enabling Hadoop framework to process big data in real time [30,34] All the research studies reported in the preceding discussions and a concise survey [35], provide a strong base to look at the problem of processing big data in real time in the form of stream processing framework. A rudimentary study was carried out in our own work [36] to present a very basic and conceptual idea of real time stream processing architecture.…”

Section: Related Workmentioning

confidence: 99%

“…Present section describes a basic real time stream processing architecture (parts of which has been described as RStreaming in [36]) which has its fundamental data structure as Digraph distributed over a cluster of nodes. Following subsections illustrate different aspects of this architecture and it is noteworthy to consider that task forking, and queuing models would be applied on this digraph architecture for further optimization of query processing qualities.…”

Section: Real Time Stream Processing Architecturementioning

confidence: 99%

See 1 more Smart Citation

Theories of Real Time Stream Processing Architecture Using Task Forking Model

Munshi¹,

Saha²,

Goswami³

2023

INDJCSE

View full text Add to dashboard Cite

Present study reports the theoretical aspects of big data stream processing architecture and it's performance metric identification, performance optimization and related experiments. The proposed architecture is considered as a complex directed graph model with various real time computational elements as nodes and big data tuples as edges, forming a real time topology. The notions of hard time deadline bound computation on streaming big data tuple along with minimum performance guarantee of processing every tuple have been introduced in the present research. Time bound computation issues in the real time stream computing architecture have been improved by optimizing time deadline management through task forking models. An algorithm for optimization of throughput has been reported and the performance metrics of the proposed system has also been identified for proper analysis on the basis of queuing theories by expressing it through appropriate Kendalls notation. Experimental results, at the end, report considerable improvement of performance of the architecture by applying optimization algorithms on a standard dataset.

show abstract