An evaluation of the Web retrieval task at the third NTCIR workshop

Abstract. This paper explores techniques that discover terms to replace given query terms from a selected subset of documents. The Internet allows access to large numbers of documents archived in digital format. However, no user can be an expert in every field, and they trouble finding the documents that suit their purposes experts when they cannot formulate queries that narrow the search to the context they have in mind. Accordingly, we propose a method for extracting terms from searched documents to replace user-provided query terms. Our results show that our method is successful in discovering terms that can be used to narrow the search.

show abstract

“…We used a document set prepared for the NTCIR3 Web Retrieval Task [3]. This set includes about ten million Web pages written in Japanese.…”

Section: Experimental Procedures and Resultsmentioning

confidence: 99%

A New Measure for Query Disambiguation Using Term Co-occurrences

Wakaki

Masada

Takasu

et al. 2006

Intelligent Data Engineering and Automated Learning – IDEAL 2006

View full text Add to dashboard Cite

show abstract

“…For the experiments, we used NW100G-01, a corpus of 100 GB web document data containing 11,038,720 web pages gathered from the '.jp' domain for the WEB Tasks of the Third and Fourth NTCIR Workshops [18] [19].…”

Section: Experiments 441 Data Setmentioning

confidence: 99%

Buildingweb page collections efficiently exploiting local surrounding pages

Wang¹,

Oyama

2009

Prog. Inform.

Self Cite

View full text Add to dashboard Cite

This paper describes a method for building a high-quality web page collection with a reduced manual assessment cost that exploits local surrounding pages. Effectiveness of the method is shown through experiments using a researcher's homepage as an example of the target categories. The method consists of two processes: rough filtering and accurate classification. In both processes, we introduce a logical page group structure concept that is represented by the relation between an entry page and its surrounding pages based on their connection type and relative URL directory level, and use the contents of local surrounding pages according to that concept. For the first process, we propose a very efficient method for comprehensively gathering all potential researchers' homepages from the web using property-based keyword lists. Four kinds of page group models (PGMs) based on the page group structure were used for merging the keywords from the surrounding pages. Although a lot of noise pages are included if we use keywords in the surrounding pages without considering the page group structure, the experimental results show that our method can reduce the increase of noise pages to an allowable level and can gather a significant number of the positive pages that could not be gathered using a single-page-based method. For the second process, we propose composing a three-grade classifier using two base classifiers: precision-assured and recall-assured. It classifies the input to assured positive, assured negative, and uncertain pages, where the uncertain pages need a manual assessment, so that the collection quality required by an application can be assured. Each of the base classifiers is further composed of a surrounding page classifier (SC) and an entry page classifier (EC). The SC selects likely component pages and the EC classifies the entry pages using information from both the entry page and the likely component pages. An evident performance improvement of the base classifiers by the introduction of the SC is shown through experiments. Then, the reduction of the number of uncertain pages is evaluated and the effectiveness of the proposed method is shown.

show abstract

“…A file including the list of links from documents in (1) to documents in (1) and (2) is also provided. Refer to the references [3] and [4] for more details.…”

Section: Document Data Setmentioning

confidence: 99%

“…It was built in the Navigational Retrieval Task 1 using the following process: (1) the document data was constructed (we used the same document data as the NTCIR-3 WEB [3,4]); (2) 300 topics were created by 11 people; (3) the document data and the topics were distributed to nine participants, and, in turn, 16 search run results were submitted by five groups and some others by the organizers; (4) relevance judgment on the run results were performed by 10 assessors; and (5) 218 topics, each of…”

Section: Introductionmentioning

confidence: 99%

The test collection for navigational retrieval on WWW data-Design and characteristics

Oyama¹,

Ishikawa²,

Eguchi³

et al. 2005

Prog. Inform.

Self Cite

View full text Add to dashboard Cite

This paper describes the design and characteristics of a test collection for navigational retrieval of WWW data that was built through the WEB Task of the Fourth NTCIR Workshop to evaluate the retrieval effectiveness of Web search systems. This reusable test collection consists of 100 gigabytes of Web document data and 300 topics of various types and corresponding relevance judgments. Among the several types of 'Navigational Retrieval,' we selected the 'Known Item Search,' which simulates a situation where a user searches for one or a few 'representative Web pages' of a known item. It is assumed that the user knows about the item but may not have seen its Web page. Relevance judgments were performed on the probable documents mainly from the viewpoint of representativeness of respective known items represented by the topics. Using the judgment results, several evaluation measures were applied to various retrieval results. Based on the evaluation results, relationships among the types of topics, Web-page styles and search methods are discussed. The stability of the evaluation results with different numbers of topics is also analyzed.

show abstract

An evaluation of the Web retrieval task at the third NTCIR workshop

Cited by 19 publications

References 3 publications

A New Measure for Query Disambiguation Using Term Co-occurrences

A New Measure for Query Disambiguation Using Term Co-occurrences

Buildingweb page collections efficiently exploiting local surrounding pages

The test collection for navigational retrieval on WWW data-Design and characteristics

Contact Info

Product

Resources

About