An architecture for the aggregation and analysis of scholarly usage data

Sompel, Herbert De; Bollen, Johan

doi:10.1145/1141753.1141821

Cited by 33 publications

(41 citation statements)

References 17 publications

(41 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This requires the aggregation of a meaningful, representative sample of the scholarly community, similar in span to the ISI IF sample, and efforts to compensate for the increased diversity of the usage data sample (e.g., excluding all agents that are not scholarly authors and taking into account particular discipline-specific demographics and preferences). This article has provided an initial exploration of the second issue whereas the architecture described by Bollen and Van de Sompel (2006b) may offer at least a technical solution to the first issue, combined with efforts to standardize the various types of online usage. Questions remain as to how one can create a truly representative usage sample of the global scholarly community.…”

Section: Discussionmentioning

confidence: 99%

“…The first dimension, span, entails the aggregation of usage data across a wide range of information services to create a more global, representative sample of the scholarly community (i.e., increase the sample's span). In fact, Bollen and Van de Sompel (2006b) proposed an architecture for the large-scale aggregation of usage data which could be employed to achieve such global samples; however, this architecture addresses only the technical issues involved in aggregating such samples; it does not address the issue of what constitutes a representative global sample or which digital information services that usage should be aggregated for. The second dimension, diversity, entails efforts to understand and control how community characteristics affect usage-based impact assessments, regardless of whether the sampled community is representative of the global scholarly community.…”

Section: Introductionmentioning

confidence: 99%

“…Whereas Bollen and Van de Sompel (2006b) focused on aspects of the first dimension (i.e., sample span), this article addresses the second dimension (i.e., sample diversity): How do the characteristics of the community for which usage was recorded affect usage-based assessments of impact? Usage of scholarly resources for all 23 California State University (CSU) campuses, comprising about 405,000 students and 44,000 faculty and staff, was recorded throughout the entire period of October 2003 to August 2005 by the CSU linking servers ( Van de Sompel & Beit-Arie, 2001).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Usage impact factor: The effects of sample characteristics on usage‐based impact metrics

Bollen

Sompel

2007

J. Am. Soc. Inf. Sci.

Self Cite

View full text Add to dashboard Cite

There exist ample demonstrations that indicators of scholarly impact analogous to the citation-based ISI Impact Factor can be derived from usage data; however, so far, usage can practically be recorded only at the level of distinct information services. This leads to communityspecific assessments of scholarly impact that are difficult to generalize to the global scholarly community. In contrast, the ISI Impact Factor is based on citation data and thereby represents the global community of scholarly authors. The objective of this study is to examine the effects of community characteristics on assessments of scholarly impact from usage. We define a journal Usage Impact Factor that mimics the definition of the Thomson Scientific ISI Impact Factor. Usage Impact Factor rankings are calculated on the basis of a large-scale usage dataset recorded by the linking servers of the California State University system from 2003 to 2005. The resulting journal rankings are then compared to the Thomson Scientific ISI Impact Factor that is used as a reference indicator of general impact. Our results indicate that the particular scientific and demographic characteristics of a discipline have a strong effect on resulting usagebased assessments of scholarly impact. In particular, we observed that as the number of graduate students and faculty increases in a particular discipline, Usage Impact Factor rankings will converge more strongly with the ISI Impact Factor.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Usage impact factor: The effects of sample characteristics on usage‐based impact metrics

Bollen

Sompel

2007

J. Am. Soc. Inf. Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Most participants in the reviewed user studies rated only a few recommendations and four studies (15 %) were conducted with fewer than five participants [62,123,171]; five studies (19 %) had five to ten participants [66,84,101]; three studies (12 %) had 11-15 participants [15,146,185]; and five studies (19 %) had 16-50 participants [44,118,121]. Six studies (23 %) were conducted with more than 50 participants [93,98,117].…”

Section: User Studiesmentioning

confidence: 99%

“…This architecture has some relevance for recommender systems, since many tasks in academic search are related to recommender systems (e.g., crawling and indexing PDFs, and matching user models or search-queries with research papers). Bollen and van de Sompel published an architecture that later served as the foundation for the research-paper recommender system bX [15]. This architecture focuses on recording, processing, and exchanging scholarly usage data.…”

Section: Datasets and Architecturesmentioning

confidence: 99%

Research-paper recommender systems: a literature survey

et al. 2015

View full text Add to dashboard Cite

In the last 16 years, more than 200 research articles were published about research-paper recommender systems. We reviewed these articles and present some descriptive statistics in this paper, as well as a discussion about the major advancements and shortcomings and an overview of the most common recommendation concepts and approaches. We found that more than half of the recommendation approaches applied content-based filtering (55 %). Collaborative filtering was applied by only 18 % of the reviewed approaches, and graph-based recommendations by 16 %. Other recommendation concepts included stereotyping, item-centric recommendations, and hybrid recommendations. The content-based filtering approaches mainly utilized papers that the users had authored, tagged, browsed, or downloaded. TF-IDF was the most frequently applied weighting scheme. In addition to simple terms, n-grams, topics, and citations were utilized to model users' information needs. Our review revealed some shortcomings of the current research. First, it remains unclear which recommendation concepts and approaches are the most promising. For instance, researchers reported different results on the performance of contentbased and collaborative filtering. Sometimes content-based filtering performed better than collaborative filtering and Linnaeus University, Kalmar, Sweden sometimes it performed worse. We identified three potential reasons for the ambiguity of the results. (A) Several evaluations had limitations. They were based on strongly pruned datasets, few participants in user studies, or did not use appropriate baselines. (B) Some authors provided little information about their algorithms, which makes it difficult to re-implement the approaches. Consequently, researchers use different implementations of the same recommendations approaches, which might lead to variations in the results. (C) We speculated that minor variations in datasets, algorithms, or user populations inevitably lead to strong variations in the performance of the approaches. Hence, finding the most promising approaches is a challenge. As a second limitation, we noted that many authors neglected to take into account factors other than accuracy, for example overall user satisfaction. In addition, most approaches (81 %) neglected the user-modeling process and did not infer information automatically but let users provide keywords, text snippets, or a single paper as input. Information on runtime was provided for 10 % of the approaches. Finally, few research papers had an impact on research-paper recommender systems in practice. We also identified a lack of authority and long-term research interest in the field: 73 % of the authors published no more than one paper on research-paper recommender systems, and there was little cooperation among different co-author groups. We concluded that several actions could improve the research landscape: developing a common evaluation framework, agreement on the information to include in research papers, a stronger focus on non-accuracy aspect...

show abstract