Efficient information gathering on the Internet

Etzioni, Oren; Hanks, Steve; Jiang, Tao; Karp, Richard M.; Madani, Omid; Waarts, Orli

doi:10.1109/sfcs.1996.548482

Cited by 39 publications

(30 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These include work by Fuhr (1999) on a decision-theoretic approach to source selection and by Voorhees (1995) on an approach to select sources and merge results based on historical data. Etzioni et al (1996) also study the optimal sequence in which to query information sources in a sequential query problem where the broker pays each information source in order to query it. Si and Callan (2004) propose a deterministic Dynamic Programming (DP) based algorithm for source selection.…”

Section: Operational Decisions In Dirmentioning

confidence: 99%

Usercentric Operational Decision Making in Distributed Information Retrieval

Hosanagar

2011

Information Systems Research

View full text Add to dashboard Cite

Information specialists in enterprises regularly use distributed information retrieval (DIR) systems that query a large number of information retrieval (IR) systems, merge the retrieved results, and display them to users. There can be considerable heterogeneity in the quality of results returned by different IR servers. Further, because different servers handle collections of different sizes and have different processing and bandwidth capacities, there can be considerable heterogeneity in their response times. The broker in the DIR system has to decide which servers to query, how long to wait for responses, and which retrieved results to display based on the benefits and costs imposed on users. The benefit of querying more servers and waiting longer is the ability to retrieve more documents. The costs may be in the form of access fees charged by IR servers or user's cost associated with waiting for the servers to respond. We formulate the broker's decision problem as a stochastic mixed-integer program and present analytical solutions for the problem. Using data gathered from FedStats-a system that queries IR engines of several U.S. federal agencies-we demonstrate that the technique can significantly increase the utility from DIR systems. Finally, simulations suggest that the technique can be applied to solve the broker's decision problem under more complex decision environments. Electronic copy available at: http://ssrn.com/abstract=926928 User-Centric Operational Decision-Making in Distributed Information RetrievalKartik Hosanagar + AbstractInformation specialists in enterprises regularly use Distributed Information Retrieval (DIR) systems that query a large number of Information Retrieval (IR) systems, merge the retrieved results and display them to users. There can be considerable heterogeneity in the quality of results returned by different IR servers. Further, since different servers handle collections of different sizes, have different processing and bandwidth capacities, there can be considerable heterogeneity in their response times. The broker in the distributed IR system has to decide which servers to query, how long to wait for responses and which retrieved results to display based on the benefits and costs imposed on users. The benefit of querying more servers and waiting longer is the ability to retrieve more documents. The costs may be in the form of access fees charged by IR servers or user's cost associated with waiting for the servers to respond. We formulate the broker's decision problem as a stochastic mixed integer program and present analytical results for the optimal query set and wait time. Using data gathered from Fedstats -a system that queries IR engines of several US federal agencies -we demonstrate that the technique can significantly increase the utility from DIR systems. Finally, we present a simulation-based optimization technique to solve the broker's decision problem under more complex decision environments. The technique is computationally efficient and can be used to ge...

show abstract

Section: Operational Decisions In Dirmentioning

confidence: 99%

Usercentric Operational Decision Making in Distributed Information Retrieval

Hosanagar

2011

Information Systems Research

View full text Add to dashboard Cite

show abstract

“…In computer science, "meta-search" is an example of a consensus problem, involving combining the results of several search engines (see [131]). Cohen, Schapire, and Singer [89] showed how meta-search can be formulated as a problem of consensus.…”

Section: Meta-search Collaborative Filtering 11mentioning

confidence: 99%

Computer science and decision theory

Roberts

2008

Ann Oper Res

View full text Add to dashboard Cite

This paper reviews applications in computer science that decision theorists have addressed for years, discusses the requirements posed by these applications that place great strain on decision theory/social science methods, and explores applications in the social and decision sciences of newer decision-theoretic methods developed with computer science applications in mind. The paper deals with the relation between computer science and decision-theoretic methods of consensus, with the relation between computer science and game theory and decisions, and with "algorithmic decision theory."

show abstract

“…), but we do not have a detailed picture of the transition between this lower bound and the one in Theorem 4.6. There are a number of promising further directions for investigation, including the analysis of query incentives for more complex queries (as Charikar et al did for a single-player case [6]), and models of interaction that incorporate response time (as Etzioni et al did for a non-network setting [8]). It is natural to conjecture that the transitional behavior at b = 2 in our model hints at a more general phenomenon related to the growth rate of the underlying network.…”

Section: Extensions and Further Directionsmentioning

confidence: 99%

Query Incentive Networks

Raghavan

2005

Advances in Computer Science – ASIAN 2005. Data Management on the Web

View full text Add to dashboard Cite

The concurrent growth of on-line communities exhibiting large-scale social structure, and of large decentralized peer-to-peer file-sharing systems, has stimulated new interest in understanding networks of interacting agents as economic systems. Here we formulate a model for query incentive networks, motivated by such systems: users seeking information or services can pose queries, together with incentives for answering them, that are propagated along paths in the network. This type of information-seeking process can be formulated as a game among the nodes in the network, and this game has a natural Nash equilibrium.In such systems, it is a fundamental question to understand how much incentive is needed in order for a node to achieve a reasonable probability of extracting an answer to a query from the network. We study the size of query incentives as a function both of the rarity of the answer and the structure of the underlying network. This leads to natural questions related to strategic behavior in branching processes. Whereas the classically studied criticality of branching processes is centered around the region where the branching parameter is 1, we show in contrast that strategic interaction in incentive propagation exhibits critical behavior when the branching parameter is 2. * Department of Computer Science, Cornell University, Ithaca NY. Supported in part by a David and Lucile Packard Foundation Fellowship and NSF grants 0329064 and 0311333. This work was performed in part while visiting Carnegie Mellon University, Pittsburgh PA.† Verity, Inc. Sunnyvale, CA.-bb-error = = Figure 1: The propagation of a query with rewards. Each node is offered a reward by its parent, and offers a (smaller) reward to its children for continued propagation. The search down each branch stops when the reward reaches 0, or when a node with an answer is reached.

show abstract

Efficient information gathering on the Internet

Cited by 39 publications

References 7 publications

Usercentric Operational Decision Making in Distributed Information Retrieval

Usercentric Operational Decision Making in Distributed Information Retrieval

Computer science and decision theory

Query Incentive Networks

Contact Info

Product

Resources

About