Algorithms for processing the group K nearest-neighbor query on distributed frameworks

Algorithms for processing closest-pairs and nearest-neighbors queries on big spatial data in parallel and distributed frameworks

“…Two different computational techniques in Reducers of Phases 2 and We introduced the "Fast Sums" technique in [53] and used it in both approaches, when iteratively calculating the sums of Euclidean or xdistances to compare them with the heap top. Instead of waiting for the whole iteration on the Query to complete and then compare the result with the heap top (as in the literature), we are checking the heap top distance in every iteration step, and if the partial sum is bigger, the loop exits.…”

“…The algorithm in [53] utilized ideas and elements from previous works, namely query definition and heuristics [60], processing without indexes and PS heuristics [65,67], reparti tioning of data [24,27]. A new high performance point/cell filtering method was introduced as well as several other calculation improvements and output data minimization techniques, which greatly improved total algorithm performance.…”

“…The algorithm presentation in this section is a unified approach of the ones presented in [52] and [53] and their differences will be noted when necessary. The algorithm modification in [55] will be presented separately, because it is quite different.…”

“…In order to improve Phase 2 performance, we had to reduce the number of eligible cells and this was the greatest improvement introduced in [53]. After several experiments we ended using the centroid as the center of a circle which is expanding itself to intersect cells with at least K Training points in total (Figure 5.11).…”

“…The algorithm was first presented in [52] and was implemented in the Apache Hadoop framework. Later in [53] an improved version of the algorithm was presented, along with a SpatialHadoop implementation and a study of what happens internally during the operation of the algorithm, using specialized cluster wide metrics. In [55] a prepartitioning method was introduced for the Training, which proved to be very effective.…”