Concurrently processing thousands of web queries, each with a response time under a fraction of a second, necessitates maintaining and operating massive data centers. For largescale web search engines, this translates into high energy consumption and a huge electric bill. This work takes the challenge to reduce the electric bill of commercial web search engines operating on data centers that are geographically far apart. Based on the observation that energy prices and query workloads show high spatio-temporal variation, we propose a technique that dynamically shifts the query workload of a search engine between its data centers to reduce the electric bill. Experiments on real-life query workloads obtained from a commercial search engine show that significant financial savings can be achieved by this technique.
Query forwarding is an important technique for preserving the result quality in distributed search engines where the index is geographically partitioned over multiple search sites. The key component in query forwarding is the thresholding algorithm by which the forwarding decisions are given. In this paper, we propose a linear-programming-based thresholding algorithm that significantly outperforms the current state-of-the-art in terms of achieved search efficiency values. Moreover, we evaluate a greedy heuristic for partial index replication and investigate the impact of result cache freshness on query forwarding performance. Finally, we present some optimizations that improve the performance further, under certain conditions. We evaluate the proposed techniques by simulations over a real-life setting, using a large query log and a document collection obtained from Yahoo!. Categories and Subject Descriptors General TermsAlgorithms, Design, Performance, Experimentation KeywordsSearch engines, distributed IR, query forwarding, optimization, linear programming, index replication, result caching BACKGROUNDCommercial web search engines of the past relied on a single search site (data center), which processed queries issued Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. from all around the world. This approach had the typical scalability problems in centralized architectures. Moreover, queries issued from distant locations suffered from poor response times as the network latency between the user and the site became an issue. For such queries, either the query processing times had to be shortened, thus degrading the result quality, or users experienced unreasonable response times, which had implications on user satisfaction [16].At this point, replicating the data (i.e., the web collection and the inverted index built upon it) over multiple, geographically distant search sites emerged as a feasible solution. In this strategy, each geographical region is mapped to a nearby search site. A search site processes over its full web index only the queries originating from the regions assigned to itself 1 . Although this strategy reduces network latencies, the scalability still remains as an issue since the entire web index had to be maintained on all search sites and queries are evaluated over the full web index.A strategy that contrasts replication is to partition the data disjointly and assign each site only the documents obtained (crawled) from its region [8]. In this strategy, local queries of a region are evaluated over the partial index in the corresponding search site. The underlying assumption here is that users are interested more in documents located...
Sparse matrix-vector multiplication (SpMxV) is a kernel operation widely used in iterative linear solvers. The same sparse matrix is multiplied by a dense vector repeatedly in these solvers. Matrices with irregular sparsity patterns make it difficult to utilize cache locality effectively in SpMxV computations. In this work, we investigate single-and multiple-SpMxV frameworks for exploiting cache locality in SpMxV computations. For the single-SpMxV framework, we propose two cache-size-aware top-down row/column-reordering methods based on 1D and 2D sparse matrix partitioning by utilizing the column-net and enhancing the row-column-net hypergraph models of sparse matrices. The multipleSpMxV framework depends on splitting a given matrix into a sum of multiple nonzero-disjoint matrices so that the SpMxV operation is performed as a sequence of multiple input-and output-dependent SpMxV operations. For an effective matrix splitting required in this framework, we propose a cache-size-aware top-down approach based on 2D sparse matrix partitioning by utilizing the row-column-net hypergraph model. The primary objective in all of the three methods is to maximize the exploitation of temporal locality. We evaluate the validity of our models and methods on a wide range of sparse matrices by performing actual runs through using OSKI. Experimental results show that proposed methods and models outperform state-of-the-art schemes.
In a shared-nothing, distributed text retrieval system, queries are processed over an inverted index that is partitioned among a number of index servers. In practice, the index is either document-based or term-based partitioned. This choice is made depending on the properties of the underlying hardware infrastructure, query traffic distribution, and some performance and availability constraints. In query processing on retrieval systems that adopt a term-based index partitioning strategy, the high communication overhead due to the transfer of large amounts of data from the index servers forms a major performance bottleneck, deteriorating the scalability of the entire distributed retrieval system. In this work, to alleviate this problem, we propose a novel inverted index partitioning model that relies on hypergraph partitioning. In the proposed model, concurrently accessed index entries are assigned to the same index servers, based on the inverted index access patterns extracted from the past query logs. The model aims tominimize the communication overhead that will be incurred by future queries while maintaining the computational load balance among the index servers. We evaluate the performance of the proposed model through extensive experiments using a real-life text collection and a search query sample. Our results show that considerable performance gains can be achieved relative to the term-based index partitioning strategies previously proposed in literature. In most cases, however, the performance remains inferior to that attained by document-based partitioning. © 2013 ACM
Abstract. A typical first step of a direct solver for linear system M x = b is reordering of symmetric matrix M to improve execution time and space requirements of the solution process. In this work, we propose a novel nesteddissection-based ordering approach that utilizes hypergraph partitioning. Our approach is based on formulation of graph partitioning by vertex separator (GPVS) problem as a hypergraph partitioning problem. This new formulation is immune to deficiency of GPVS in a multilevel framework hence enables better orderings. In matrix terms, our method relies on the existence of a structural factorization of the input M matrix in the form of M = AA T (or M = AD 2 A T ). We show that the partitioning of the row-net hypergraph representation of rectangular matrix A induces a GPVS of the standard graph representation of matrix M . In the absence of such factorization, we also propose simple, yet effective structural factorization techniques that are based on finding an edge clique cover of the standard graph representation of matrix M , and hence applicable to any arbitrary symmetric matrix M . Our experimental evaluation has shown that the proposed method achieves better ordering in comparison to state-ofthe-art graph-based ordering tools even for symmetric matrices where structural M = AA T factorization is not provided as an input. For matrices coming from linear programming problems, our method enables even faster and better orderings.Key words. Fill-reducing ordering; hypergraph partitioning; combinatorial scientific computing.AMS subject classifications. 05C65, 05C85, 68R10, 68W051. Introduction. In most scientific computing applications, the core of the computation is solving a symmetric system of linear equations in the form M x = b . Direct methods, such as LU and Cholesky factorizations, are commonly preferred for solving such systems for their numerical robustness. A typical first step of a direct method is a heuristic reordering of the rows and columns of M to reduce fill in the triangular factor matrices. The fill is the set of zero entries in M that become nonzero in the triangular factor matrices. Another goal in reordering is to reduce the number of floating-point operations required to perform the triangular factorization, also known as operation count. It is equal to the sum of the squares of the nonzeros of each eliminated row/column, hence it is directly related with the number of fills.For a symmetric matrix, the evolution of the nonzero structure during the factorization can easily be described in terms of its graph representation [42]. In graph terms, the elimination of a vertex (which corresponds to a row/column of the matrix) creates edges for every pair of its adjacent vertices. In other words, elimination of a vertex makes its adjacent vertices clique of size its degree minus one. In this process, the added edges directly correspond to the fill in the matrix. Obviously, the amount of fill and operation count depends on the row/column elimination order. The aim of ordering is to...
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.