An Initiative to Track Sentiments in Altmetrics

Halevi, Gali; Schimming, Laura

doi:10.29024/joa.1

Cited by 11 publications

(6 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extracted citation statements are classified into supporting, disputing, or mentioning, in order to identify studies that have tested the claim and to evaluate how a scientific claim has been evaluated in the literature by subsequent research. We emphasize that scite is not doing sentiment analysis ( 31, 22, 32, 33 ), where a subjective polarity is associated with a claim, but a discrete classification into three discursive functions relative to the scientific debate (see Murray et al ( 34 ) for an example of previous work with typing citations based on rhetorical intention). We consider that for capturing the reliability of a claim, a classification decision into supporting or disputing must be evidence-based, backed by scientific arguments.…”

Section: Methodsmentioning

confidence: 99%

scite: a smart citation index that displays the context of citations and classifies their intent using deep learning

Nicholson¹,

Mordaunt²,

Lopez³

et al. 2021

Preprint

View full text Add to dashboard Cite

Citation indices are tools used by the academic community for research and research evaluation which aggregate scientific literature output and measure scientific impact by collating citation counts. Citation indices help measure the interconnections between scientific papers but fall short because they only display paper titles, authors, and the date of publications, and fail to communicate contextual information about why a citation was made. The usage of citations in research evaluation without due consideration to context can be problematic, if only because a citation that disputes a paper is treated the same as a citation that supports it. To solve this problem, we have used machine learning and other techniques to develop a "smart citation index" called scite, which categorizes citations based on context. Scite shows how a citation was used by displaying the surrounding textual context from the citing paper, and a classification from our deep learning model that indicates whether the statement provides supporting or disputing evidence for a referenced work, or simply mentions it. Scite has been developed by analyzing over 23 million full-text scientific articles and currently has a database of more than 800 million classified citation statements. Here we describe how scite works and how it can be used to further research and research evaluation.

show abstract

Section: Methodsmentioning

confidence: 99%

scite: a smart citation index that displays the context of citations and classifies their intent using deep learning

Nicholson¹,

Mordaunt²,

Lopez³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“… 2 , 70 , 72 – 74 While Altmetrics currently does not analyze the sentiment generated by an article, there are efforts in exploring how to incorporate this data toward the AAS. 75 Finally AAS is also dependent on social media platforms for the dissemination of scientific articles online and would have been limited to news outlet or academic journal websites as the means of generating attention. Articles published after 2004 would be more likely to have higher AAS because this was when Facebook was founded, and subsequently would provide a greater platform for disseminating scientific research.…”

Section: Limitationsmentioning

confidence: 99%

The 50 most impactful articles on the medial ulnar collateral ligament: An altmetric analysis of online media

et al. 2022

View full text Add to dashboard Cite

The purpose of this study was to use the Altmetric Attention Score to determine the 50 most impactful medial ulnar collateral ligament articles in online media and compare their characteristics to the most-cited medial ulnar collateral ligament articles in the scientific literature. The Altmetric database was queried to identify all published articles about the medial ulnar collateral ligament, and this list was stratified by the Altmetric Attention Score to identify the 50 highest scoring articles. Several data elements were extracted, including article topic, article type, journal name, and the number of online mentions on Facebook, Twitter, news, and other platforms. Each article’s geographic origin was determined based on the institutional affiliation of the first author. Our index search yielded 1283 articles published between 1987 and 2020, from which the 50 articles with the highest Altmetric Attention Scores were included for analysis. Altmetric Attention Scores of the top 50 medial ulnar collateral ligament articles ranged from 20 to 482 (median: 32, interquartile range: 20–62). The most common article type was original research (72%), and the most common topic was epidemiology/risk factors (26%). A majority of studies were Level 3 (36%) or Level 4 evidence (36%). Of the top 50 medial ulnar collateral ligament articles, 94% originated from the United States. A few articles had a high Altmetric Attention Score, suggesting that medial ulnar collateral ligament research does not generate consistently high online attention. The lack of Level 1 studies suggests the need for high-level studies on the medial ulnar collateral ligament. Most studies originated in the United States and were published in the American Orthopaedic Society for Sports Medicine–affiliated journals. The medial ulnar collateral ligament articles included in this study differed substantially from a previous report of the most-cited medial ulnar collateral ligament articles in the literature, suggesting that alternative metrics add a unique dimension to understanding the overall impact of published research on the medial ulnar collateral ligament.

show abstract

“…Following this approach, a subjective polarity can be associated with a citation to try to capture an opinion about the cited paper. The evidence used for sentiment classification rely on the presence of affective words in the citation context, with an associated polarity score capturing the strength of the affective state (Athar, 2014;Halevi & Schimming, 2018;Hassan et al, 2018;Yousif et al, 2019;). Yan et al (2020), for instance, uses a generic method called SenticNet to identify sentiments in citation contexts extracted from PubMed Central XML files, without particular customization to the scientific domain (only a preprocessing to remove the technical terms from the citation contexts is applied).…”

Section: Task Modeling and Training Datamentioning

confidence: 99%

scite: A smart citation index that displays the context of citations and classifies their intent using deep learning

Nicholson¹,

Mordaunt²,

Lopez³

et al. 2021

Quantitative Science Studies

View full text Add to dashboard Cite

Citation indices are tools used by the academic community for research and research evaluation which aggregate scientific literature output and measure impact by collating citation counts. Citation indices help measure the interconnections between scientific papers but fall short because they fail to communicate contextual information about a citation. The usage of citations in research evaluation without consideration of context can be problematic, because a citation that presents contrasting evidence to a paper is treated the same as a citation that presents supporting evidence. To solve this problem, we have used machine learning, traditional document ingestion methods, and a network of researchers to develop a “smart citation index” called scite, which categorizes citations based on context. Scite shows how a citation was used by displaying the surrounding textual context from the citing paper and a classification from our deep learning model that indicates whether the statement provides supporting or contrasting evidence for a referenced work, or simply mentions it. Scite has been developed by analyzing over 25 million full-text scientific articles and currently has a database of more than 880 million classified citation statements. Here we describe how scite works and how it can be used to further research and research evaluation. Peer Review https://publons.com/publon/10.1162/qss_a_00146

show abstract

An Initiative to Track Sentiments in Altmetrics

Cited by 11 publications

References 22 publications

scite: a smart citation index that displays the context of citations and classifies their intent using deep learning

scite: a smart citation index that displays the context of citations and classifies their intent using deep learning

The 50 most impactful articles on the medial ulnar collateral ligament: An altmetric analysis of online media

scite: A smart citation index that displays the context of citations and classifies their intent using deep learning

Contact Info

Product

Resources

About