Efficient Subgraph Matching on Large RDF Graphs Using MapReduce

Abstract

With the popularity of knowledge graphs growing rapidly, large amounts of RDF graphs have been released, which raises the need for addressing the challenge of distributed subgraph matching queries. In this paper, we propose an efficient distributed method to answer subgraph matching queries on big RDF graphs using MapReduce. In our method, query graphs are decomposed into a set of stars that utilize the semantic and structural information embedded RDF graphs as heuristics. Two optimization techniques are proposed to further improve the efficiency of our algorithms. One algorithm, called RDF property filtering, filters out invalid input data to reduce intermediate results; the other is to improve the query performance by postponing the Cartesian product operations. The extensive experiments on both synthetic and real-world datasets show that our method outperforms the close competitors S2X and SHARD by an order of magnitude on average.

Survey of RDF Stores & SPARQL Engines for Querying Knowledge Graphs

“…HBase Sempala [170] 2014 HDFS SemStore [206] 2014 TripleBit SparkRDF [41] 2014 HDFS/Spark TriAD [70] 2014 Custom CK15 [42] 2015 Custom CliqueSquare [59] 2015 HDFS DREAM [72] 2015 RDF-3X AdPart [73] 2016 Custom DiploCloud [210] 2016 Custom gStore-D [151] 2016 gStore JARS [159] 2016 Custom S2RDF [171] 2016 HDFS S2X [168] 2016 Spark SPARQLGX [60] 2016 HDFS Wukong [178] 2016 DrTM-KV Koral [92] 2017 Custom SANSA [114] 2017 HDFS Spartex [4] 2017 GSP/Custom Stylus [80] 2017 Trinity Neptune 2018 Custom PRoST [47] 2018 HDFS RDFox-D [156] 2018 RDFox WORQ [123] 2018 Spark DiStRDF [202] 2019 HDFS gStore-D2 [150] 2019 Custom Leon [67] 2019 Custom SPT+VP [79] 2019 Spark StarMR [199] 2019 HDFS DISE [89] 2020 Spark DP2RPQ [200] 2020 Spark Triag [136] 2020 Spark WISE [68] 2020 Leon or keyword searches). Dynamic programming is used for join reordering in order to optimize the query.…”

“…SPARQL query, joins are reordered based on global statistics, with the property tables used for s-s joins and vertical partitioning used for other joins. The query is then translated into Spark SQL for execution.StarMR[199] (2019) is a distributed RDF store that centers around optimizations for star joins. A graphbased storage scheme is employed, where for each node in the graph, its outward edges are represented in an adjacency list; this then supports efficient evaluation for s-s star joins.…”

A Survey of RDF Stores & SPARQL Engines for Querying Knowledge Graphs

“…HBase Sempala [170] 2014 HDFS SemStore [206] 2014 TripleBit SparkRDF [41] 2014 HDFS/Spark TriAD [70] 2014 Custom CK15 [42] 2015 Custom CliqueSquare [59] 2015 HDFS DREAM [72] 2015 RDF-3X AdPart [73] 2016 Custom DiploCloud [210] 2016 Custom gStore-D [151] 2016 gStore JARS [159] 2016 Custom S2RDF [171] 2016 HDFS S2X [168] 2016 Spark SPARQLGX [60] 2016 HDFS Wukong [178] 2016 DrTM-KV Koral [92] 2017 Custom SANSA [114] 2017 HDFS Spartex [4] 2017 GSP/Custom Stylus [80] 2017 Trinity Neptune 2018 Custom PRoST [47] 2018 HDFS RDFox-D [156] 2018 RDFox WORQ [123] 2018 Spark DiStRDF [202] 2019 HDFS gStore-D2 [150] 2019 Custom Leon [67] 2019 Custom SPT+VP [79] 2019 Spark StarMR [199] 2019 HDFS DISE [89] 2020 Spark DP2RPQ [200] 2020 Spark Triag [136] 2020 Spark WISE [68] 2020 Leon Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 7 April 2021 doi:10.20944/preprints202104.0199.v1…”

“…These secondary indexes can also be used for computing semi-join reductions on individual machines, thus reducing network traffic. StarMR [199] (2019) is a distributed RDF store that centers around optimizations for star joins. A graphbased storage scheme is employed, where for each node in the graph, its outward edges are represented in an adjacency list; this then supports efficient evaluation for s-s star joins.…”

NetNCSP: Nonoverlapping closed sequential pattern mining

“…For example, <a(abc)(ac)c(cd)> is a sequence in traditional SPM, since (abc) is an element in the sequence. In gap pattern in a sequence, which is pattern matching problem [24][25][26]. Thus, gap constraint is also called a wildcard gap or flexible wildcards [27,28] in pattern matching fields.…”