Neural network applications for automatic new topic identification on excite web search engine data logs

Abstract

The analysis of contextual information in search engine query logs is an important, yet difficult task. Users submit few queries, and search multiple topics sometimes with closely related context. Identification of topic changes within a search session is an important branch of contextual information analysis. The purpose of this study is to propose a topic identification algorithm using neural networks. A sample from the Excite data log i s selected to train the neural network and then the neural network is used to identify topic changes in the data log. As a result, 76% of topic shifts and 92% of topic continuations are identified correctly.

Investigating the performance of automatic new topic identification across multiple datasets

“…The classification of the search patterns is based on terms of the consecutive queries within a session. The categorization of time interval and search pattern is selected similar to those of (Ozmutlu andCavdur, 2005a, 2005b), Ozmutlu (2006), Ozmutlu, et al (2004a) to avoid any bias during comparison.…”

“…Their approach was replicated on Excite search engine data (Ozmutlu and Cavdur, 2005a). Ozmutlu and Cavdur (2005b) and Ozmutlu, et al (2004a) proposed an artificial neural network to automatically identify topic changes, and showed that neural networks successfully provided new topic identification. Application of neural networks for automatic new topic identification does not contain semantic analysis, and relies on the statistical characteristics of the queries.…”

Defining Topic Boundaries in Search Engine Transaction Logs Using Genetic Algorithms

“…The main finding of these studies was that the idea of using query patterns and time-intervals in identifying topic shifts is valuable, but can be improved. In order to improve the performance of new topic identification Ozmutlu et al (2004a) and Ozmutlu and Cavdur (2005b) applied artificial neural networks to define boundaries of separate topics in the same session, and identified topic shifts fairly successfully. Ozmutlu (2006) found that the effect of the characteristics of web search queries on the occurrence of topic shifts/continuations is statistically significant.…”

“…Several nonsemantic methodologies, primarily based on learning algorithms, have been previously applied for new topic identification and session identification, such as the Demspter-Shafer Theory (Ozmutlu and Cavdur 2005a;Ozmutlu, Cavdur, and Ozmutlu 2006), neural networks (Ozmutlu and Cavdur 2005b;Ozmutlu, Cavdur, Ozmutlu, and Spink 2004a), but room for improvement exists. A new nonsemantic methodology for fast and successful automatic topic identification would be a valuable contribution to the field of information science.…”

Automatic Topic Identification for Idea Summarization in Idea Visualization Programs

“…He et al [7] proposed using an unsupervised clustering technique.Özmutlu et al [8] proposed the use of a neural network classification technique. Clifton et al [9] proposed the use of an association rule mining technique.…”