Query performance for tightly coupled distributed digital libraries

Ribeiro-Neto, Berthier; Barbosa, Ramurti A.

doi:10.1145/276675.276695

Cited by 61 publications

(63 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The previous work for distributing the inverted index over a collection of servers is focused on the local and global inverted files strategies [13], [15], showing that the local inverted file is a more balanced strategy and a good query throughput could be achieved in most cases.…”

Section: Related Workmentioning

confidence: 99%

“…The novelty of this work relies on the size of the collection represented (1TB of text) and the large number of workstations simulated. This work is especially related to [3] and [13], but it differs mainly in three points. First, the probabilistic model is considered and therefore, disjunctive queries are used in the system (without the reduction in the answer set provided by the conjunctive operations).…”

Section: Related Workmentioning

confidence: 99%

“…One strategy is to partition the document collection so that each query server is responsible for a disjoint subset of documents in the collection (called local inverted files in [13]). The other option is to partition based on the index terms so that each query server stores inverted lists corresponding to only a subset of the index terms in the collection (called global inverted files in [13]). The study in [15] indicates that the local inverted file organization uses system resources effectively, provides good query throughput and is more resilient to failures.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Performance Analysis of Distributed Architectures to Index One Terabyte of Text

Cacheda

Plachouras

Ounis

2004

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract.We simulate different architectures of a distributed Information Retrieval system on a very large Web collection, in order to work out the optimal setting for a particular set of resources. We analyse the effectiveness of a distributed, replicated and clustered architecture using a variable number of workstations. A collection of approximately 94 million documents and 1 terabyte of text is used to test the performance of the different architectures. We show that in a purely distributed architecture, the brokers become the bottleneck due to the high number of local answer sets to be sorted. In a replicated system, the network is the bottleneck due to the high number of query servers and the continuous data interchange with the brokers. Finally, we demonstrate that a clustered system will outperform a replicated system if a large number of query servers is used, mainly due to the reduction of the network load.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Performance Analysis of Distributed Architectures to Index One Terabyte of Text

Cacheda

Plachouras

Ounis

2004

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The distributed query processing on text databases has been extensively studied, being the inverted index one of the most used data structures [63,57,37,4,11,32,42,47,66]. Those works obtain highly efficient inverted indexes, with a high query throughput.…”

Section: Previous Related Work 131 Distributed Query Processing Anmentioning

confidence: 99%

Distributed text search using suffix arrays

Arroyuelo¹,

Bonacic²,

Gil-Costa³

et al. 2014

Parallel Computing

View full text Add to dashboard Cite

Text search is a classical problem in Computer Science, with many data-intensive applications. For this problem, suffix arrays are among the most widely known and used data structures, enabling fast searches for phrases, terms, substrings and regular expressions in large texts. Potential application domains for these operations include large-scale search services, such as Web search engines, where it is necessary to efficiently process intensive-traffic streams of on-line queries. This paper proposes strategies to enable such services by means of suffix arrays. We introduce techniques for deploying suffix arrays on clusters of distributed-memory processors and then study the processing of multiple queries on the distributed data structure. Even though the cost of individual search operations in sequential (non-distributed) suffix arrays is low in practice, the problem of processing multiple queries on distributed-memory systems, so that hardware resources are used efficiently, is relevant to services aimed at achieving high query throughput at low operational costs. Our theoretical and experimental performance studies show that our proposals are suitable solutions for building efficient and scalable on-line search services based on suffix arrays. IntroductionIn the last decade, the design of efficient data structures and algorithms for textual databases and related applications has received a great deal of attention, due to the rapid growth of the amount of text data available from different sources. Typical applications support text searches over big text collections in a client-server fashion, where the user queries are answered by a dedicated server [15]. The server efficiency-in terms of running time-is of paramount importance in cases where the services demanded by clients generate a heavy work load. A feasible way to overcome the limitations of sequential computers is to resort to the use of several computers, or processors, which work together to serve the ever increasing client demands [19].One such approach to efficient parallelization is to distribute the data onto the processors, in such a way that it becomes feasible to exploit locality via parallel processing of user requests, each on a subset of the data. As opposed to shared-memory models, this distributed-memory model provides the benefit of better * Corresponding author. Address: Av. España 1680, Valparaíso, Chile. Phone: +56 2 432 6722. Fax: +56 2 432 6702. , in distributed memory systems, and describes strategies to reduce the inter-processor communication and to improve the load balance at search time. Indexed Text SearchingThe advent of powerful processors and cheap storage has enabled alternative models for information retrieval, other than the traditional one of a collection of documents indexed by a fixed set of keywords. One is the full text model, in which the user expresses its information need via words, phrases or patterns to be matched for, and the information system retrieves those documents containing the user-specified pa...

show abstract

“…Current digital libraries are becoming more complex systems which include text search, functionality relating to hypertext, multimedia, the Web and highly interactive interfaces [20]. If we have problems producing good web sites as evidenced by much research done to address problems on the Web [26], then it would not be unreasonable to anticipate that we will have problems creating good digital libraries!…”

Section: Problems With Digital Librariesmentioning

confidence: 99%

Design Guidelines and User-Centred Digital Libraries

Theng¹,

Duncker²,

Mohd-Nasir³

et al. 1999

Research and Advanced Technology for Digital Libraries

View full text Add to dashboard Cite

Abstract. As current digital libraries are becoming more complex, the facilities provided by them will increase and the difficulty of learning associated with the complexity of using these facilities will also increase. In order to produce usable and useful interactive systems, designers need to ensure that good design features are incorporated into the systems, taking into consideration end-users' needs and cultural backgrounds. We carried out a study to investigate useful design features digital libraries should have. The study provides insights on the usability impact of digital libraries for task completion and end-users' perceived impressions on the effectiveness of the digital libraries. The results also suggest that there is little provision on the interface to cater to end-users' browsing and inter-cultural needs. Hence, this paper also discusses design guidelines for the design of user-centred digital libraries.

show abstract

Query performance for tightly coupled distributed digital libraries

Cited by 61 publications

References 16 publications

Performance Analysis of Distributed Architectures to Index One Terabyte of Text

Performance Analysis of Distributed Architectures to Index One Terabyte of Text

Distributed text search using suffix arrays

Design Guidelines and User-Centred Digital Libraries

Contact Info

Product

Resources

About