Handling distributed XML queries over large XML data based on MapReduce framework

Fan, Hongjie; Ma, Zhiyi; Wang, Dianhui; Liu, Junfei

doi:10.1016/j.ins.2018.04.028

Cited by 12 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to index models, many query evaluation algorithms have been developed in recent years to address complex queries. Basic XML queries include single-Path and Twig-Path queries [37]. Most of the existing query evaluation methods encode the nodes of the XML map using a regional labeling scheme and then simply cluster the labels using the same tag name.…”

Section: Indexing and Query Evaluationmentioning

confidence: 99%

UCIS-X: An Updatable Compact Indexing Scheme for Efficient Extensible Markup Language Document Updating and Query Evaluation

Hsu

Liao

2020

IEEE Access

View full text Add to dashboard Cite

One of the difficulties faced when using XML as the data storage structure is query inefficiency. Therefore, various indexing methods have been proposed. When designing indexing methods, the first step is to choose the labeling method. Some labeling methods can work well; however, if they cannot effectively support update operations, their use is subject to considerable limitations. Most of the update-friendly labeling methods proposed in the literature assign a unique label to each node in XML and provide an expandable mechanism for future insertion. However, they encounter some difficulties, such as increasing the index space, more difficulty in evaluating the relationships between nodes, and increasing the complexity of labels. In this paper, we introduce a novel update-friendly labeling scheme called branch map, which records the correspondence between parent and child nodes instead of assigning a label to each node. The space required for the index is reduced considerably. More importantly, the branch map can maintain the profile as if it was encoded initially, even after being frequently updated. This paper also proposes a compact indexing scheme called UCIS-X. Experimental results indicate that UCIS-X performs well in terms of index size, query, and update efficiency.

show abstract

Section: Indexing and Query Evaluationmentioning

confidence: 99%

UCIS-X: An Updatable Compact Indexing Scheme for Efficient Extensible Markup Language Document Updating and Query Evaluation

Hsu

Liao

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The large amounts of XML data created daily have influenced the development of Big Data solutions to handle massive XML data in a scalable and efficient environment ( Boussaid et al, 2006 ; Fan et al, 2018 ). Nowadays, a multitude of published studies have proposed methodologies and solutions for processing these types of files using Big Data tools.…”

Section: Introductionmentioning

confidence: 99%

Efficient processing of complex XSD using Hive and Spark

Martinez-Mosquera

Navarrete

Luján‐Mora

2021

PeerJ Computer Science

View full text Add to dashboard Cite

The eXtensible Markup Language (XML) files are widely used by the industry due to their flexibility in representing numerous kinds of data. Multiple applications such as financial records, social networks, and mobile networks use complex XML schemas with nested types, contents, and/or extension bases on existing complex elements or large real-world files. A great number of these files are generated each day and this has influenced the development of Big Data tools for their parsing and reporting, such as Apache Hive and Apache Spark. For these reasons, multiple studies have proposed new techniques and evaluated the processing of XML files with Big Data systems. However, a more usual approach in such works involves the simplest XML schemas, even though, real data sets are composed of complex schemas. Therefore, to shed light on complex XML schema processing for real-life applications with Big Data tools, we present an approach that combines three techniques. This comprises three main methods for parsing XML files: cataloging, deserialization, and positional explode. For cataloging, the elements of the XML schema are mapped into root, arrays, structures, values, and attributes. Based on these elements, the deserialization and positional explode are straightforwardly implemented. To demonstrate the validity of our proposal, we develop a case study by implementing a test environment to illustrate the methods using real data sets provided from performance management of two mobile network vendors. Our main results state the validity of the proposed method for different versions of Apache Hive and Apache Spark, obtain the query execution times for Apache Hive internal and external tables and Apache Spark data frames, and compare the query performance in Apache Hive with that of Apache Spark. Another contribution made is a case study in which a novel solution is proposed for data analysis in the performance management systems of mobile networks.

show abstract

Section: Introductionmentioning

confidence: 99%

“…For example, reference [13] uses WordNet, an external corpus, and the top retrieved documents as data sources for QE. Some of the other research works for query processing based on hybrid resources are [14,24,18]. Among the above data sources, Wikipedia and WordNet are popular choices for semantic enrichment of the initial query [44,35,2].…”

Section: Introductionmentioning

confidence: 99%

A new approach for query expansion using Wikipedia and WordNet

Azad

Deepak

2019

Information Sciences

View full text Add to dashboard Cite

Query expansion (QE) is a well-known technique used to enhance the effectiveness of information retrieval. QE reformulates the initial query by adding similar terms that help in retrieving more relevant results. Several approaches have been proposed in literature producing quite favorable results, but they are not evenly favorable for all types of queries (individual and phrase queries). One of the main reasons for this is the use of the same kind of data sources and weighting scheme while expanding both the individual and the phrase query terms. As a result, the holistic relationship among the query terms is not well captured or scored. To address this issue, we have presented a new approach for QE using Wikipedia and WordNet as data sources. Specifically, Wikipedia gives rich expansion terms for phrase terms, while WordNet does the same for individual terms.We have also proposed novel weighting schemes for expansion terms: in-link score (for terms extracted from Wikipedia) and a tf-idf based scheme (for terms extracted from WordNet). In the proposed Wikipedia-WordNet-based QE technique (WWQE), we weigh the expansion terms twice: first, they are scored by the weighting scheme individually, and then, the weighting scheme scores the selected expansion terms concerning the entire query using correlation score. The proposed approach gains improvements of 24% on the MAP score and 48% on the GMAP score over unexpanded queries on the FIRE dataset. Experimental results achieve a significant improvement over individual expansion and other related state-of-the-art approaches. We also analyzed the effect on retrieval effectiveness of the proposed technique by varying the number of expansion terms. makes information processing a big challenge and creates a vocabulary gap between user queries and indexed documents. It is common for a user's query Q and its set of relevant documents D to use different vocabulary and language styles while referring to the same concept. For example, terms 'buy' and 'purchase' have the same meaning; however, only one of these can be present in the documents' index while the other one can be the user's query term. This makes it difficult to retrieve the information actually wanted by the user [21,28]. An effective strategy to fill this gap is to use the Query Expansion (QE) technique, which enhances the retrieval effectiveness by adding expansion terms to the initial query. Selection of the expansion terms plays a crucial role in QE because only a small subset of the expanded terms are actually relevant to the query [26].In this sense, the approach for selection of expansion terms is equally important in comparison to what we do further with the expanded terms in order to retrieve the desired information. QE has a long research history in information retrieval (IR) [31,39]. It has the potential to enhance IR's effectiveness by adding relevant terms that can help to discriminate the relevant documents from irrelevant ones. The source of expansion terms plays a significant role in QE. A variety of ...

show abstract

Handling distributed XML queries over large XML data based on MapReduce framework

Cited by 12 publications

References 20 publications

UCIS-X: An Updatable Compact Indexing Scheme for Efficient Extensible Markup Language Document Updating and Query Evaluation

UCIS-X: An Updatable Compact Indexing Scheme for Efficient Extensible Markup Language Document Updating and Query Evaluation

Efficient processing of complex XSD using Hive and Spark

A new approach for query expansion using Wikipedia and WordNet

Contact Info

Product

Resources

About