An Efficient Distributed SPARQL Query Processing Scheme Considering Communication Costs in Spark Environments

Lim, Jongtae; Kim, Byoung-Hoon; Lee, Hyeonbyeong; Choi, Do Young; Bok, Kyoungsoo; Yoo, Jaesoo

doi:10.3390/app12010122

Cited by 3 publications

(3 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figs. [2][3][4][5][6][7][8][9][10][11] show the relationship between the delivered numbers of frames as a function of waiting time, with N=20000. Figure . 12 Relationship between maximum successfully transmitted frames and waiting time Fig.…”

Section: Effect Of Waiting Time (Tw)mentioning

confidence: 99%

See 1 more Smart Citation

Effect of Waiting Time and Number of Slots on Vehicular Networking Employing Slotted ALOHA Protocol

2024

IJIES

View full text Add to dashboard Cite

This work investigates through simulation effect of waiting time and number of slots on the effectiveness of using Slotted ALOHA in vehicular connectivity as a random access technique. The objective is to reach optimum level of throughput under the vehicular dynamic environment. The presented work proposes to concentrate on waiting time as a controlling parameter that can be used in an adaptive manner to enable throughput optimization in response to changing vehicular density. The work proved through MATLAB simulation the importance of waiting time parameter in controlling number of delivered frames and throughput. The results show that based on the exponential relationship between throughput and data traffic, there is a logarithmic relationship between data traffic and waiting time, which complements another logarithmic relationship between throughput and waiting time. The simulation also shows that there is an inverse relationship between throughput and data traffic as a function of set waiting time. The simulation shows that there is an exponential relationship between waiting time and maximum number of delivered frames. The work shows that a linear region of operating Slotted ALOHA can be specified, such that an optimum and dynamically adjusted waiting time is set with overall effect on number of slots and throughput. The dynamic shape function of the curve correlating waiting time to available number of slots is mathematically modelled and differentially controlled, thus an optimizing and controlling algorithm is produced, which improves significantly the conventional ALOHA performance and has an advantage over other used ALOHA derivatives, in that it concentrates on available slots and waiting time, which greatly contributes towards reduction in collisions and improving of throughput.

show abstract

Section: Effect Of Waiting Time (Tw)mentioning

confidence: 99%

“…This will contribute towards safer, easier, and more reliable and comfortable driving. In addition, vehicular connectivity will result in lower fuel consumption and will assist in reducing traffic congestion [4,5].…”

Section: Introductionmentioning

confidence: 99%

Effect of Waiting Time and Number of Slots on Vehicular Networking Employing Slotted ALOHA Protocol

2024

IJIES

View full text Add to dashboard Cite

show abstract

“…The more data you filter, the lower the cost will be. Lim et al [25] studied all possible query execution paths in grouping subquery computation overhead and selected effective query execution paths through efficient query algorithms to reduce the cost. Path analysis technology is also applied in all walks of life.…”

Section: Optimization Of Project Analysis 41 Cost Optimization Estima...mentioning

confidence: 99%

Optimization of the Join between Large Tables in the Spark Distributed Framework

Wang

2023

Applied Sciences

View full text Add to dashboard Cite

The Join task between Spark large tables takes a long time to run and produces a lot of disk I/O, network I/O and disk occupation in the Shuffle process. This paper proposes a lightweight distributed data filtering model that combines broadcast variables and accumulators using RoaringBitmap. When the data in the two tables are not exactly matched, the dimension table Key is collected through the accumulator, compressed by RoaringBitmap and distributed to each node using broadcast variables. The distributed fact table data can be pre-filtered on the local server, which effectively reduces the data transmission and disk reading and writing in the Shuffle phase. Experimental results show that this optimization method can reduce disk usage, shorten the running time and reduce network I/O and disk I/O for Spark Join tasks in the case of massive data, and the effect is more obvious when the two tables have a higher incomplete matching degree or a fixed matching degree but a larger amount of data. This optimization scheme has the advantages of being easy to use, being easy to maintain and having an obvious effect, and it can be applied to many development scenarios.

show abstract

An efficient and scalable SPARQL query processing framework for big data using MapReduce and hybrid optimum load balancing

Kumar,

P.S.

2023

Data & Knowledge Engineering

View full text Add to dashboard Cite

An Efficient Distributed SPARQL Query Processing Scheme Considering Communication Costs in Spark Environments

Cited by 3 publications

References 38 publications

Effect of Waiting Time and Number of Slots on Vehicular Networking Employing Slotted ALOHA Protocol

Effect of Waiting Time and Number of Slots on Vehicular Networking Employing Slotted ALOHA Protocol

Optimization of the Join between Large Tables in the Spark Distributed Framework

An efficient and scalable SPARQL query processing framework for big data using MapReduce and hybrid optimum load balancing

Contact Info

Product

Resources

About