Document keyphrases as subject metadata: incorporating document key concepts in search results

Abstract

Most search engines display some document metadata, such as title, snippet and URL, in conjunction with the returned hits to aid users in determining documents. However, metadata is usually fragmented pieces of information that, even when combined, does not provide an overview of a returned document. In this paper, we propose a mechanism of enriching metadata of the returned results by incorporating automatically extracted document keyphrases with each returned hit. We hypothesize that keyphrases of a document can better represent the major theme in that document. Therefore, by examining the keyphrases in each returned hit, users can better predict the content of documents and the time spent on downloading and examining the irrelevant documents will be reduced substantially.

Automatic keyphrase annotation of scientific documents using Wikipedia and genetic algorithms

“…This is because despite the large overlap between MA and VK, there is not overlap between MA and HA 1 . Whereas when additional gold standards, HA 2 and HA 3 , are available and used for evaluating the overall inter-indexer consistency score of the machine annotator, a more accurate estimation of the machine annotator's performance could be achieved. In fact, as can be seen in the illustration, if the quality of MA and HA 1 were to be compared with each other by using HA 2 and HA 3 as gold standards, the overall quality of MA would be significantly higher than HA 1 Whereas, in case of the former, additional sets of keyphrases are usually not available and need to be created manually, for example the small wiki-20 dataset used in this work has taken ninety man-hours to create.…”

“…Annotating scientific documents with keyphrases as subject/topical metadata helps both humans and information retrieval systems to focus their search and discovery efforts on the most relevant items of interest and reduces the recall effort (i.e., ratio of desired to examined) [2,3]. However, despite the fact that authors of scientific literature, especially those published in journals and conference proceedings, are encouraged and often required by editors to provide a list of keyphrases, scientific documents with manually assigned keyphrases by either authors or professional annotators are still in the minority.…”

“…Nevertheless, we believe deploying these two approaches together would improve the overall performance of keyphrase indexing, as they estimate the semantic relatedness of topics very differently using two independent information sources in Wikipedia and therefore could complement each other. We measure the category-based relatedness of two Wikipedia topics as: (2) where D is the maximum depth of the taxonomy, i.e., 16 in case of the Wikipedia dump used in this work. The distance function returns the length of the shortest path between topic 1 and topic 2 in terms of the number of nodes along the path.…”

Automatic Keyphrase Extraction from Medical Documents

“…We treat the noun phrases in the document as the candidate keyphrases [1]. To identify the noun phrases, documents should be tagged.…”

“…A number of previous works has suggested that document keyphrases can be useful in a various applications such as retrieval engines [1], [2], [3], browsing interfaces [4], thesaurus construction [5], and document classification and clustering [6].…”