Hourly analysis of a very large topically categorized web query log

Abstract

We review a query log of hundreds of millions of queries that constitute the total query traffic for an entire week of a generalpurpose commercial web search service. Previously, query logs have been studied from a single, cumulative view. In contrast, our analysis shows changes in popularity and uniqueness of topically categorized queries across the hours of the day. We examine query traffic on an hourly basis by matching it against lists of queries that have been topically pre-categorized by human editors. This represents 13% of the query traffic. We show that query traffic from particular topical categories differs both from the query stream as a whole and from other categories. This analysis provides valuable insight for improving retrieval effectiveness and efficiency. It is also relevant to the development of enhanced query disambiguation, routing, and caching algorithms.

Workload Modeling for Computer Systems Performance Evaluation

“…However, when the workload includes items whose duration is on the scale of hours (such as parallel jobs), the daily cycle cannot be ignored [248]. Also, in some cases it is wrong to assume that the daytime workload is more important; for example, in web search workloads the activity of home users in the evening and at night is no less important than the activity of professionals during the day [63]. Over long ranges, a nonstationary workload can be the result of changing usage patterns as users get to know the system better.…”

“…The diurnal cycle is a well-known and widely recognized phenomenon, which appears in practically all workloads (e.g., [326,320,524,63,761,745,756,381,192,42]). An example from parallel supercomputers is shown in Figure 6.34.…”

“…For example, many people start their day with reading and responding to email or reading the news, but do this less often later in the day [381]. An interesting study of web search behavior showed that the popularity of different search topics changes at different times of day, with music searches peaking at 3-4 AM, porn at 5-6 AM, and personal finance at 8-10 AM [63]. The largest shifts in popularity in video-on-demand also occur during the early morning hours [745].…”

“…[326,320,524,63,761,745,756,381,192]). Activity patterns typically peak during the day in the morning and afternoon and are low at night.…”

“…• Different topics are popular at different times of the day: music peaks at 3-4 AM, porn at 5-6 AM, and personal finance at 8-10 AM [63].…”