Dataflow query execution in a parallel main-memory environment

Wilschut, A.N.; Apers, Peter M. G.

doi:10.1109/pdis.1991.183069

Cited by 123 publications

(118 citation statements)

References 16 publications

Supporting

Mentioning

118

Contrasting

Order By: Relevance

“…The Symmetric Hash Join (SHJ) algorithm [14,15] is a stream join that extends the original hash join algorithm. It produces join output as early as possible, i.e.…”

Section: Related Workmentioning

confidence: 99%

“…Both the front-stage as well as MESHJOIN use hash joins, so the CMESHJOIN algorithm can overall be seen as possessing two complementary hash join phases, somewhat similar to Symmetric Hash Join [14,15]. The MESHJOIN phase uses the master data relation R as the probe input, with the largest part of R typically being stored on disk.…”

Section: Cmeshjoin (Cached Mesh Join)mentioning

confidence: 99%

See 1 more Smart Citation

Skewed distributions in semi-stream joins: How much can caching help?

Naeem

Dobbie

Lutteroth

et al. 2017

Information Systems

View full text Add to dashboard Cite

Semi-stream join algorithms join a fast data stream with a disk-based relation. This is important, for example, in real-time data warehousing where a stream of transactions is joined with master data before loading it into a data warehouse. In many important scenarios, the stream input has a skewed distribution, which makes certain performance optimisations possible.We propose two such optimisation techniques: 1) a caching technique for frequently used master data, and 2) a technique for selective load shedding of stream tuples. The caching technique is fine-grained, operating on a tuple-level. Furthermore, it is generic in the sense that it can be applied to different semistream join algorithms to deal with high loads. We analyze it by combining it with various well-known semi-stream joins, and show that it improves the service rate by more than 40% for typical data with skewed distributions. The load shedding technique sheds the fraction of the stream that is most expensive to join. In contrast to existing approaches, the service rate improves under load shedding. We present experimental data showing significant improvements as compared to related approaches and perform a sensitivity analysis for various internal parameters.

show abstract

“…The Symmetric Hash Join (SHJ) algorithm [14,15] is a stream join that extends the original hash join algorithm. It produces join output as early as possible, i.e.…”

Section: Related Workmentioning

confidence: 99%

Section: Cmeshjoin (Cached Mesh Join)mentioning

confidence: 99%

Skewed distributions in semi-stream joins: How much can caching help?

Naeem

Dobbie

Lutteroth

et al. 2017

Information Systems

View full text Add to dashboard Cite

show abstract

“…-Index Nested-Loop Join [4]: Each R-tuple is read in turn and its joint attribute(s) are then used to search an index on S and retrieve the tuples that match. -Symmetric Hash Join (Pipelined Hash Join) [16]: Each tuple from either R or S is read in turn and is both stored in the hash table for R (or S, respectively) and used to to probe the hash table for S (or R, respectively). Any matching R-(or S-) tuples (respectively) are retrieved.…”

Section: Technical Contextmentioning

confidence: 99%

A Foundation for the Replacement of Pipelined Physical Join Operators in Adaptive Query Processing

Eurviriyanukul

Fernandes

Paton

2006

Current Trends in Database Technology – EDBT 2006

View full text Add to dashboard Cite

Abstract. Adaptive query processors make decisions as to the most effective evaluation strategy for a query based on feedback received while the query is being evaluated. In essence, any of the decisions made by the optimizer (e.g., on operator order or on which operators to use) may be revisited in an adaptive query processor. This paper focuses on changes to physical operators (e.g., the specific join operators used, such as hash-join or merge-join) in pipelined query evaluators. In so doing, the paper characterizes the runtime properties of pipelined operators in a way that makes explicit when specific operators may be replaced, and that allows the validity of operator replacements to be proved. This is illustrated with reference to the substitution of join operators during their evaluation.

show abstract

“…The Tukwila [66] and Niagara projects [67] introduce data integration systems with adaptive query processing and XML query operator implementations that efficiently support pipelining. Pipelining of hash joins is discussed in [68,69,70]. Pipelining is often also termed streaming or non-blocking execution.…”

Section: Ldap and Mdsmentioning

confidence: 99%

Peer‐To‐Peer Grid Databases for Web Service Discovery

Hoschek

2003

Grid Computing

View full text Add to dashboard Cite

Dataflow query execution in a parallel main-memory environment

Cited by 123 publications

References 16 publications

Skewed distributions in semi-stream joins: How much can caching help?

Skewed distributions in semi-stream joins: How much can caching help?

A Foundation for the Replacement of Pipelined Physical Join Operators in Adaptive Query Processing

Peer‐To‐Peer Grid Databases for Web Service Discovery

Contact Info

Product

Resources

About