Reviewing Natural Language Processing Research

Cohen, K. Bretonnel; Fort, Karën; Mieskes, Margot; Névéol, Aurélie

doi:10.18653/v1/2020.acl-tutorials.4

Cited by 4 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A way forward Short of changing the incentive structure, we can do more to strengthen the review process. Tutorials on reviewing (Cohen et al, 2020) and the implementation of their recommendations would go a long way toward ensuring that we maintain a high standard at a high volume.…”

Section: Publication Biasmentioning

confidence: 99%

On the Gap between Adoption and Understanding in NLP

Bianchi¹,

Hovy²

2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

There are some issues with current research trends in NLP that can hamper the free development of scientific research. We identify five of particular concern: 1) the early adoption of methods without sufficient understanding or analysis; 2) the preference for computational methods regardless of risks associated with their limitations; 3) the resulting bias in the papers we publish; 4) the impossibility of re-running some experiments due to their cost; 5) the dangers of unexplainable methods. If these issues are not addressed, we risk a loss of reproducibility, reputability, and subsequently public trust in our field. In this position paper, we outline each of these points and suggest ways forward.

show abstract

Section: Publication Biasmentioning

confidence: 99%

On the Gap between Adoption and Understanding in NLP

Bianchi¹,

Hovy²

2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

show abstract

“…The direct beneficiaries of sentiment are online shoppers, even capital investors, politicians, campaign managers, public relations firms, and marketing managers. In recent years, enterprises have improved as a result of SA applications 7–10 …”

Section: Introductionmentioning

confidence: 99%

“…In recent years, enterprises have improved as a result of SA applications. [7][8][9][10] Political choices, social events, financial services, health services, and consumer products are a few of the applications that rely on SA. 11 A wide range of applications relies on deep learning (DL) that accomplished better results in different fields including emotion detection, artificial intelligence 12 , and signal processing.…”

Section: Introductionmentioning

confidence: 99%

FBO‐RNN: Fuzzy butterfly optimization‐basedRNN‐LSTMfor extracting sentiments from Twitter Emoji database

Venkataraman

Mohandoss

2023

Concurrency and Computation

View full text Add to dashboard Cite

Social media networks have seen a significant increase in purpose and scale. Sentimental analysis is a component of social networking platforms that uses shared material to infer information about individual sentiments and emotions. In recent years, sentiment analysis (SA) research has grown in popularity. Twitter is the most popular social media platform, with users from many languages and cultures participating. Emojis are used by users to express themselves, and social media platforms contain a wide range of symbols, emotions, and opinions. A novel framework for SA based on Emojis is presented in this article. Initially, the noise-free videos and images are filtered. The dictionary of Jieba was obtained by adding the English Emoji lexicon and English Internet slang lexicon to segment English text. Initially, the Emojis are converted into textual features.Different sentiment classes such as positive, very positive, neutral, negative, and very negative classes are classified using long short-term memory (LSTM) in the recurrent neural network (RNN)-based Fuzzy Butterfly Optimization (FBO) algorithm. The freeware WEKA software tool with different evaluation measures performs experimental investigations. Ultimately, the proposed model demonstrates superior results in the case of SA than other state-of-the-art methods.

show abstract

Peer review analyze: A novel benchmark resource for computational analysis of peer reviews

et al. 2022

View full text Add to dashboard Cite

Peer Review is at the heart of scholarly communications and the cornerstone of scientific publishing. However, academia often criticizes the peer review system as non-transparent, biased, arbitrary, a flawed process at the heart of science, leading to researchers arguing with its reliability and quality. These problems could also be due to the lack of studies with the peer-review texts for various proprietary and confidentiality clauses. Peer review texts could serve as a rich source of Natural Language Processing (NLP) research on understanding the scholarly communication landscape, and thereby build systems towards mitigating those pertinent problems. In this work, we present a first of its kind multi-layered dataset of 1199 open peer review texts manually annotated at the sentence level (∼ 17k sentences) across the four layers, viz. Paper Section Correspondence, Paper Aspect Category, Review Functionality, and Review Significance. Given a text written by the reviewer, we annotate: to which sections (e.g., Methodology, Experiments, etc.), what aspects (e.g., Originality/Novelty, Empirical/Theoretical Soundness, etc.) of the paper does the review text correspond to, what is the role played by the review text (e.g., appreciation, criticism, summary, etc.), and the importance of the review statement (major, minor, general) within the review. We also annotate the sentiment of the reviewer (positive, negative, neutral) for the first two layers to judge the reviewer’s perspective on the different sections and aspects of the paper. We further introduce four novel tasks with this dataset, which could serve as an indicator of the exhaustiveness of a peer review and can be a step towards the automatic judgment of review quality. We also present baseline experiments and results for the different tasks for further investigations. We believe our dataset would provide a benchmark experimental testbed for automated systems to leverage on current NLP state-of-the-art techniques to address different issues with peer review quality, thereby ushering increased transparency and trust on the holy grail of scientific research validation. Our dataset and associated codes are available at https://www.iitp.ac.in/~ai-nlp-ml/resources.html#Peer-Review-Analyze.

show abstract

Reviewing Natural Language Processing Research

Cited by 4 publications

References 18 publications

On the Gap between Adoption and Understanding in NLP

On the Gap between Adoption and Understanding in NLP

FBO‐RNN: Fuzzy butterfly optimization‐basedRNN‐LSTMfor extracting sentiments from Twitter Emoji database

Peer review analyze: A novel benchmark resource for computational analysis of peer reviews

Contact Info

Product

Resources

About