An effectiveness analysis of altmetrics indices for different levels of artificial intelligence publications

Zhang, Xi; Wang, Xianhai; Zhao, Hongke; Pablos, Patricia Ordóñez de; Sun, Yongqiang; Xiong, Hui

doi:10.1007/s11192-019-03088-x

Cited by 28 publications

(13 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…7 The chord graph of cross-linguistic-border research leadership flows citation-based h-index, Altmetrics, and Altmetrics-based h-index) and compare its performance with that of other conventional indices. ROC curve analysis is a graphic method to illustrate the performance of a binary classification system under different recognition thresholds (Hassan et al, 2017;Zhang et al, 2019). The larger AUC (area under the ROC curve) is, the better performance the focal binary classification system has.…”

Section: Discussionmentioning

confidence: 99%

Characterizing research leadership on geographically weighted collaboration network

Zhang

2021

Scientometrics

View full text Add to dashboard Cite

Research collaborations, especially long-distance and international collaborations, have become increasingly prevalent worldwide. Recent studies highlighted the significant role of research leadership in collaborations. However, existing measures of the research leadership do not take into account the intensity of leadership in the co-authorship network. More importantly, the spatial features, which influence the collaboration patterns and research outcomes, have not been incorporated in measuring the research leadership. To fill the gap, we construct an institution-level weighted co-authorship network that integrates two types of weight on the edges: the intensity of collaborations and the spatial score (the geographical distance adjusted by the cross-linguistic-border nature). Based on this network, we propose a novel metric, namely the spatial research leadership rank, to identify the leading institutions while considering both the collaboration intensity and the spatial features. The leadership of an institution is measured by the following three criteria: (a) the institution frequently plays the corresponding rule in papers with other institutions; (b) the institution frequently plays the corresponding rule in longer distance and even cross-linguistic-border collaborations; (c) the participating institutions led by the institution have high leadership status themselves. Harnessing a dataset of 323,146 journal publications in pharmaceutical sciences during 2010-2018, we perform a comprehensive analysis of the geographical distribution and dynamic patterns of research leadership flows at the institution level. The results demonstrate that the SpatialLeaderRank outperforms baseline metrics in predicting the scholarly impact of institutions. And the result remains robust in the field of Information Science and Library Science.

show abstract

Section: Discussionmentioning

confidence: 99%

Characterizing research leadership on geographically weighted collaboration network

Zhang

2021

Scientometrics

View full text Add to dashboard Cite

show abstract

“…in social sciences (Ringelhan, Wollersheim & Welpe, 2015) -in comparison with Twitter which, according to several studies, is a more representative source for altmetrics due to a quicker outreach of scientific works shared on this medium (Xia et al, 2016), but also due to a higher correlation with citations (Haustein, Costas & Lariviére, 2015). On the other hand, the academic network Mendeley reflects a kind of impact that is closer to citations and which has been documented by their mutual significant correlation (Alhoori & Furuta, 2014;Mohammadi et al, 2014;Mohammadi & Thelwall, 2014;Bornmann, 2015;Maflahi & Thelwall, 2016;Amath, 2017;Zhang, 2019). It is interesting to note that Mohammadi and Thelwall (2014) found the highest level of correspondence in natural sciences and related areas, and the lowest level in humanities -according to their findings, bookmarking is a tool for discovering connections between various disciplines and revealing potential interdisciplinary relations and interactions, so it has a descriptor function.…”

Section: Previous Research In Altmetricsmentioning

confidence: 94%

Bibliometrics versus altmetrics: Researchers’ attitudes in Slovakia

Michaela

Ondrišová

Šušol

2021

Iberoam J Sci Meas Comm

View full text Add to dashboard Cite

Objective. The paper presents principal theoretical starting points and an overview of attitudes to altmetrics-based research evaluation among scientists of various disciplines in Slovakia, more specifically in comparison with traditional, bibliometric approaches. Design/Methodology/Approach. We used a questionnaire and asked researchers from various disciplines about their knowledge of principle characteristics defining the altmetrics and about their attitudes towards the application of bibliometrics and altmetrics approaches in the processes of research evaluation. The questionnaire covered a wide range of issues, predominantly associated with interpretation and perception of the importance of alternative indicators, including the factors influencing the interpretation. In this analysis, we concentrate on those parts of research reflecting the willingness of respondents to accept alternative indicators as part of research/science assessment. Results/Discussion. The attitude to the application of alternative indicators in research assessment is not significantly more negative among Slovak scientists if we compare it to other countries. The citations are still a respected indicator perceived as a tool for creating bonds within science. If they are perceived as obsolete or insufficient, it relates to the experience of researchers whose results are predominantly aimed at non-academic target group or if they are active in the research of social phenomena and problems. Due to pragmatic reasons connected with research financing, however, registering the citations is a generally accepted priority. Familiarity and practical experience with social media were identified as one of the factors influencing the interpretation of alternative indicators in the environment of both public and academic media. Conclusions. Traditional bibliometric methods, especially in the area of research evaluation, are still preferred over the altmetrics. The scientists realize that the kind of attention captured by the indicators based on social media is not comparable with the citation-based impact. Or, more precisely, that altmetrics reflects a different kind of impact whose correspondence with a traditional world of bibliographic citations remains to be explored. Originality/Value. The article presents the results of original research carried out in the community of scientists in Slovakia, applying quantitative research methods. It contributes to the results of previous studies on the attitudes and acceptance of altmetrics in the research community, with a special focus on research assessment.

show abstract

“…Altmetrics are non-traditional metrics that cover not just citation counts but also downloads, social media shares, and other measures of the impact of research outputs [65]. Altmetrics measures the broader impact of research on society [5,7,8,[66][67][68][69][70], especially by using a much wider set of resources, including social media posts, press releases, news articles, and political debates stemming from academic work, as well as assesses wider non-academic impact [6]. In 2014, Bornmannanalyzed the advantages and disadvantages of measuring the impact of using altmetrics [4].…”

Section: Altmetricsmentioning

confidence: 99%

Measuring researchers’ potential scholarly impact with structural variations: Four types of researchers in information science (1979–2018)

2020

View full text Add to dashboard Cite

We propose a method to measure the potential scholarly impact of researchers based on network structural variations they introduced to the underlying author co-citation network of their field. We applied the method to the information science field based on 91,978 papers published between 1979 and 2018 from the Web of Science. We divided the entire period into eight consecutive intervals and measured structural variation change rates (ΔM) of individual authors in corresponding author co-citation networks. Four types of researchers are identified in terms of temporal dynamics of their potential scholarly impact-1) Increasing, 2) Decreasing, 3) Sustained, and 4) Transient. The study contributes to the understanding of how researchers' scholarly impact might evolve in a broad context of the corresponding research community. Specifically, this study illustrated a crucial role played by structural variation metrics in measuring and explaining the potential scholarly impact of a researcher. This method based on the structural variation analysis offers a theoretical framework and a practical platform to analyze the potential scholarly impact of researchers and their specific contributions.

show abstract

An effectiveness analysis of altmetrics indices for different levels of artificial intelligence publications

Cited by 28 publications

References 47 publications

Characterizing research leadership on geographically weighted collaboration network

Characterizing research leadership on geographically weighted collaboration network

Bibliometrics versus altmetrics: Researchers’ attitudes in Slovakia

Measuring researchers’ potential scholarly impact with structural variations: Four types of researchers in information science (1979–2018)

Contact Info

Product

Resources

About